- ghc-commits - Haskell.org

[Git][ghc/ghc][wip/romes/async-rpc] 2 commits: remoteAsyncReq cleans up a pendingReply on exception.
by Andrea Vezzosi (＠trac-Saizan) 29 Oct '26

29 Oct '26

Andrea Vezzosi pushed to branch wip/romes/async-rpc at Glasgow Haskell Compiler / GHC Commits: 512a412e by Andrea Vezzosi at 2026-10-29T11:09:42+01:00 remoteAsyncReq cleans up a pendingReply on exception. - - - - - 9fee3f58 by Andrea Vezzosi at 2026-10-29T13:15:16+01:00 remoteAsyncReq: send cancel if exception during writePipe fixes gap in exception coverage. - - - - - 1 changed file: - libraries/ghci/GHCi/Message/Async.hs Changes: ===================================== libraries/ghci/GHCi/Message/Async.hs ===================================== @@ -164,13 +164,15 @@ writePipe p msg = do var <- atomically $ writePipeAsync p msg either throwIO pure =<< atomically (takeTMVar var) +-- | Enqueues the message for writing. +-- +-- The returned TMVar can be checked for the outcome. writePipeAsync :: Pipe -> IddMsg -> STM (TMVar (Either SomeException ())) writePipeAsync Pipe{..} msg = do var <- newEmptyTMVar readTVar pipeWriteClosed >>= \case - True -> throwSTM PipeWriteClosedException - False -> pure () - writeTQueue (writeQueue pipeAsyncMngr) (msg, var) + True -> putTMVar var (Left $ toException PipeWriteClosedException) + False -> writeTQueue (writeQueue pipeAsyncMngr) (msg, var) pure var @@ -217,11 +219,12 @@ withLock Pipe{..} = bracket (takeMVar pipeLock) (putMVar pipeLock) . const -- * Matching requests with async replies -------------------------------------------------------------------------------- --- | Send an asynchronous request and block waiting for a matching reply +-- | Allocates a new 'pendingReply' and takes care of deallocation if the +-- continuation throws an exception. -- --- See Note [Exceptions during serialization] about exceptions the 'Put' argument might raise. -remoteAsyncReq :: Pipe -> Put -> (Maybe InboxId) -> IO (Either SomeException LB.ByteString) -remoteAsyncReq pipe msg ii = do +-- Will throw 'PipeReadClosedException' if that is the case. +newPendingReply :: Pipe -> (Int -> TMVar (Either SomeException LB.ByteString) -> IO b) -> (IO b) +newPendingReply pipe k = do let pendingReplies = pendingReply (pipeAsyncMngr pipe) @@ -229,33 +232,39 @@ remoteAsyncReq pipe msg ii = do uq <- freshId (pipeAsyncMngr pipe) wv <- newEmptyTMVarIO - isClosed <- atomically $ do - isClosed <- readTVar (pipeReadClosed pipe) - -- don't send the req if we won't get a reply. - if isClosed - then pure True - else do - modifyTVar pendingReplies (IntMap.insert uq wv) - pure False + bracketOnError + (atomically $ do + readTVar (pipeReadClosed pipe) >>= \case + -- don't set a reply as pending if we won't receive it. + True -> throwSTM PipeReadClosedException + False -> modifyTVar pendingReplies (IntMap.insert uq wv)) - when isClosed $ do - throwIO PipeReadClosedException + (const $ atomically $ modifyTVar pendingReplies (IntMap.delete uq)) + + (const $ k uq wv) + +-- | Send an asynchronous request and block waiting for a matching reply +-- +-- See Note [Exceptions during serialization] about exceptions the 'Put' argument might raise. +remoteAsyncReq :: Pipe -> Put -> (Maybe InboxId) -> IO (Either SomeException LB.ByteString) +remoteAsyncReq pipe msg ii = newPendingReply pipe $ \ uq wv -> + -- We might send a Cancel even if the request never got sent, but it will just + -- get ignored in that case. + handle (\e -> sendCancelMaybe uq e >> throwIO e) $ do -- Write request writePipe pipe (Request uq ii (runPut msg)) - `onException` - -- the req wasn't done, or no one will be reading the reply: - -- in either case, drop it from the pending map. - atomically (modifyTVar pendingReplies (IntMap.delete uq)) -- Block waiting for reply atomically (takeTMVar wv) - `onException` - atomically (do - modifyTVar pendingReplies (IntMap.delete uq) - -- we don't wait for Cancel to be written on the pipe, - -- and won't get a reply: nowhere to report it anyway. - writePipeAsync pipe (Cancel uq)) + where + sendCancelMaybe uq e + | Just PipeWriteClosedException <- fromException e = pure () + | otherwise = sendCancel uq + sendCancel uq = void . atomically $ do + -- we don't wait for Cancel to be written on the pipe, + -- and won't get a reply: nowhere to report it anyway. + writePipeAsync pipe (Cancel uq) -- | Receive a single asynchronous request and send a matching reply -- asynchronously (the handler may fork a computation that eventually replies View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/9eb22ea6ddea408cc8cfd628b0c190… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/9eb22ea6ddea408cc8cfd628b0c190… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0

[Git][ghc/ghc][wip/jeltsch/ghc-9-14-building-base] Make the Linux job not use the Docker image
by Wolfgang Jeltsch (＠jeltsch) 29 Oct '26

29 Oct '26

Wolfgang Jeltsch pushed to branch wip/jeltsch/ghc-9-14-building-base at Glasgow Haskell Compiler / GHC Commits: f1fc6af9 by Wolfgang Jeltsch at 2026-10-29T14:17:43+02:00 Make the Linux job not use the Docker image - - - - - 1 changed file: - .gitlab-ci.yml Changes: ===================================== .gitlab-ci.yml ===================================== @@ -1163,7 +1163,7 @@ base-with-other-ghcs-linux: - job: release-x86_64-linux-deb13-release optional: true dependencies: null - image: "registry.gitlab.haskell.org/ghc/ci-images/x86_64-linux-deb13:$DOCKER_REV" + image: null tags: - x86_64-linux script: .gitlab/base-ci.sh x86_64-deb13-linux 9.14.1 View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/f1fc6af9ac7fbaa7c34e4e1c0bc8cbd… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/f1fc6af9ac7fbaa7c34e4e1c0bc8cbd… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0

[Git][ghc/ghc][wip/dcoutts/io-manager-feature-bitset] In GHC.Internal.Conc.IO more uniform dispatch to alternatives
by Duncan Coutts (＠dcoutts) 29 Oct '26

29 Oct '26

Duncan Coutts pushed to branch wip/dcoutts/io-manager-feature-bitset at Glasgow Haskell Compiler / GHC Commits: 4e7cb00e by Duncan Coutts at 2026-10-29T11:39:45+00:00 In GHC.Internal.Conc.IO more uniform dispatch to alternatives Reduce and consolotate the amount of CPP used to select between different implementations of the I/O manager actions. In particular, bring the Windows and non-Windows alternatives closer: 1. by importing the modules with the same qualified as name 2. in a couple cases by providing no-op versions of APIs so that we have a more uniform API between the two Also introduce a HAVE_IOMGR_IN_LIB define, and use that to have a static guard over the (dynamic guard) to call in-lib I/O manager functions. The pattern ends up (mostly uniformly) as: blahIOManager #if defined(HAVE_IOMGR_IN_LIB) | iomgrInLib = IOMgr.blahIOManager #endif | otherwise = ... -- the in-RTS version #endif Note: * HAVE_IOMGR_IN_LIB is a static choice * iomgrInLib is a dynamic choice Where previously there was a _lot_ more ad-hoc CPP. In particular this now covers JS and wasm more consistently, by saying they do _not_ define HAVE_IOMGR_IN_LIB, and thus only ever call the in-RTS variants. And while we're at it, make threadWaitRead, threadWaitWrite fail more politely on Windows, by throwing an exception rather than terminating the RTS! On wasm, this decreases the amount of the base library object code that gets linked into a static hello world program, because the threaded I/O manager is dead code that can be avoided by the linker. ------------------------- Metric Decrease: size_hello_artifact size_hello_artifact_gzip size_hello_unicode size_hello_unicode_gzip ------------------------- - - - - - 3 changed files: - libraries/ghc-internal/src/GHC/Internal/Conc/IO.hs - libraries/ghc-internal/src/GHC/Internal/Conc/Windows.hs - libraries/ghc-internal/src/GHC/Internal/Event/Thread.hs Changes: ===================================== libraries/ghc-internal/src/GHC/Internal/Conc/IO.hs ===================================== @@ -67,36 +67,47 @@ import qualified GHC.Internal.Foreign.C.Error as C import GHC.Internal.Foreign.C.Types (CInt) import GHC.Internal.Maybe (Maybe(Nothing)) import GHC.Internal.STM as STM -import GHC.Internal.Prim (Int#, State#, RealWorld, - delay#, waitRead#, waitWrite#, raiseIO#) +import GHC.Internal.Prim (Int#, State#, RealWorld, delay#, raiseIO#) import GHC.Internal.Real ( fromIntegral ) import GHC.Internal.System.Posix.Types -#if !defined(javascript_HOST_ARCH) -import GHC.Internal.IO.SubSystem -#endif - #if defined(mingw32_HOST_OS) -import qualified GHC.Internal.Conc.Windows as Windows import GHC.Internal.Conc.Windows (asyncRead, asyncWrite, asyncDoProc, asyncReadBA, asyncWriteBA, ConsoleEvent(..), win32ConsoleHandler, toWin32ConsoleEvent) -#elif !defined(javascript_HOST_ARCH) -import qualified GHC.Internal.Event.Thread as Event +#else +import GHC.Internal.Prim (waitRead#, waitWrite#) +#endif + +-- A switch to control support for calling in-library I/O managers. +-- Plausible future directions: hoist this to .cabal file feature flag. +#if defined(javascript_HOST_ARCH) +-- Turned off for JS (which only has a single in-RTS I/O manager). +#elif defined(wasm32_HOST_ARCH) +-- Turned off for wasm (which only has in-RTS I/O managers). +#else +-- all other platforms +#define HAVE_IOMGR_IN_LIB +#endif + +#if defined(HAVE_IOMGR_IN_LIB) +import GHC.Internal.IO.SubSystem +#if defined(mingw32_HOST_OS) +import qualified GHC.Internal.Conc.Windows as IOMgr +#else +import qualified GHC.Internal.Event.Thread as IOMgr +#endif #endif #if defined(wasm32_HOST_ARCH) -import qualified GHC.Internal.Wasm.Prim.Conc as Wasm -import qualified GHC.Internal.Wasm.Prim.Flag as Wasm +import qualified GHC.Internal.Wasm.Prim.Conc as Wasm (threadDelay) +import qualified GHC.Internal.Wasm.Prim.Flag as Wasm (isJSFFIUsed) #endif ensureIOManagerIsRunning :: IO () -#if !defined(mingw32_HOST_OS) && !defined(javascript_HOST_ARCH) -ensureIOManagerIsRunning - | iomgrInLib = Event.ensureIOManagerIsRunning -#elif !defined(javascript_HOST_ARCH) ensureIOManagerIsRunning - | iomgrInLib = Windows.ensureIOManagerIsRunning +#if defined(HAVE_IOMGR_IN_LIB) + | iomgrInLib = IOMgr.ensureIOManagerIsRunning #endif | otherwise = return () @@ -107,15 +118,15 @@ ensureIOManagerIsRunning -- @since base-4.15 interruptIOManager :: IO () interruptIOManager -#if defined(mingw32_HOST_OS) - | iomgrInLib = Windows.interruptIOManager +#if defined(HAVE_IOMGR_IN_LIB) + | iomgrInLib = IOMgr.interruptIOManager #endif | otherwise = return () ioManagerCapabilitiesChanged :: IO () ioManagerCapabilitiesChanged -#if !defined(mingw32_HOST_OS) && !defined(javascript_HOST_ARCH) - | iomgrInLib = Event.ioManagerCapabilitiesChanged +#if defined(HAVE_IOMGR_IN_LIB) + | iomgrInLib = IOMgr.ioManagerCapabilitiesChanged #endif | otherwise = return () @@ -126,14 +137,20 @@ ioManagerCapabilitiesChanged -- while this thread was blocked. To safely close a file descriptor -- that has been used with 'threadWaitRead', use 'closeFdWith'. threadWaitRead :: Fd -> IO () +#if !defined(mingw32_HOST_OS) threadWaitRead fd -#if !defined(mingw32_HOST_OS) && !defined(javascript_HOST_ARCH) - | iomgrInLib = Event.threadWaitRead fd +#if defined(HAVE_IOMGR_IN_LIB) + | iomgrInLib = IOMgr.threadWaitRead fd #endif | otherwise = IO $ \s -> case fromIntegral fd of { I# fd# -> case waitRead# fd# s of { s' -> (# s', () #) }} +#else +threadWaitRead _fd + = errorWithoutStackTrace "threadWaitRead: not available on Windows" +#endif + -- | Block the current thread until data can be written to the -- given file descriptor (GHC only). @@ -142,14 +159,19 @@ threadWaitRead fd -- while this thread was blocked. To safely close a file descriptor -- that has been used with 'threadWaitWrite', use 'closeFdWith'. threadWaitWrite :: Fd -> IO () +#if !defined(mingw32_HOST_OS) threadWaitWrite fd -#if !defined(mingw32_HOST_OS) && !defined(javascript_HOST_ARCH) - | iomgrInLib = Event.threadWaitWrite fd +#if defined(HAVE_IOMGR_IN_LIB) + | iomgrInLib = IOMgr.threadWaitWrite fd #endif | otherwise = IO $ \s -> case fromIntegral fd of { I# fd# -> case waitWrite# fd# s of { s' -> (# s', () #) }} +#else +threadWaitWrite _fd + = errorWithoutStackTrace "threadWaitWrite: not available on Windows" +#endif -- | Returns an STM action that can be used to wait for data -- to read from a file descriptor. The second returned value @@ -157,8 +179,8 @@ threadWaitWrite fd -- in the file descriptor. threadWaitReadSTM :: Fd -> IO (STM.STM (), IO ()) threadWaitReadSTM fd -#if !defined(mingw32_HOST_OS) && !defined(javascript_HOST_ARCH) - | iomgrInLib = Event.threadWaitReadSTM fd +#if defined(HAVE_IOMGR_IN_LIB) && !defined(mingw32_HOST_OS) + | iomgrInLib = IOMgr.threadWaitReadSTM fd #endif | otherwise = do m <- STM.newTVarIO False @@ -176,8 +198,8 @@ threadWaitReadSTM fd -- in the file descriptor. threadWaitWriteSTM :: Fd -> IO (STM.STM (), IO ()) threadWaitWriteSTM fd -#if !defined(mingw32_HOST_OS) && !defined(javascript_HOST_ARCH) - | iomgrInLib = Event.threadWaitWriteSTM fd +#if defined(HAVE_IOMGR_IN_LIB) && !defined(mingw32_HOST_OS) + | iomgrInLib = IOMgr.threadWaitWriteSTM fd #endif | otherwise = do m <- STM.newTVarIO False @@ -209,8 +231,8 @@ closeFdWith :: (Fd -> IO ()) -- ^ Low-level action that performs the real close. -> Fd -- ^ File descriptor to close. -> IO () closeFdWith close fd -#if !defined(mingw32_HOST_OS) && !defined(javascript_HOST_ARCH) - | iomgrInLib = Event.closeFdWith close fd +#if defined(HAVE_IOMGR_IN_LIB) && !defined(mingw32_HOST_OS) + | iomgrInLib = IOMgr.closeFdWith close fd #endif | otherwise = close fd @@ -226,12 +248,10 @@ closeFdWith close fd -- Consider using @Control.Concurrent.Thread.Delay.delay@ from @unbounded-delays@ package. threadDelay :: Int -> IO () threadDelay time -#if defined(mingw32_HOST_OS) - | iomgrInLib = Windows.threadDelay time +#if defined(HAVE_IOMGR_IN_LIB) + | iomgrInLib = IOMgr.threadDelay time #elif defined(wasm32_HOST_ARCH) | Wasm.isJSFFIUsed = Wasm.threadDelay time -#elif !defined(javascript_HOST_ARCH) - | iomgrInLib = Event.threadDelay time #endif | otherwise = IO $ \s -> case time of { I# time# -> @@ -247,12 +267,10 @@ threadDelay time -- registerDelay :: Int -> IO (TVar Bool) registerDelay _usecs -#if defined(mingw32_HOST_OS) - | iomgrInLib = Windows.registerDelay _usecs -#elif !defined(javascript_HOST_ARCH) - | iomgrInLib = Event.registerDelay _usecs +#if defined(HAVE_IOMGR_IN_LIB) + | iomgrInLib = IOMgr.registerDelay _usecs #endif - | otherwise = errorWithoutStackTrace "registerDelay: requires -threaded" + | otherwise = errorWithoutStackTrace "registerDelay: requires -threaded" -- --------------------------------------------------------------------------- ===================================== libraries/ghc-internal/src/GHC/Internal/Conc/Windows.hs ===================================== @@ -24,6 +24,7 @@ module GHC.Internal.Conc.Windows #else ( ensureIOManagerIsRunning , interruptIOManager + , ioManagerCapabilitiesChanged -- * Waiting , threadDelay @@ -41,7 +42,7 @@ module GHC.Internal.Conc.Windows , module GHC.Internal.Event.Windows.ConsoleEvent ) where -import GHC.Internal.Base (($)) +import GHC.Internal.Base (($), return) import qualified GHC.Internal.Conc.POSIX as POSIX import qualified GHC.Internal.Event.Windows.Thread as WINIO import GHC.Internal.Event.Windows.ConsoleEvent @@ -122,4 +123,8 @@ ensureIOManagerIsRunning = POSIX.ensureIOManagerIsRunning interruptIOManager :: IO () interruptIOManager = POSIX.interruptIOManager <!> WINIO.interruptIOManager +-- No-op. Here to provide a uniform API to call in GHC.Internal.Conc.IO +ioManagerCapabilitiesChanged :: IO () +ioManagerCapabilitiesChanged = return () + #endif ===================================== libraries/ghc-internal/src/GHC/Internal/Event/Thread.hs ===================================== @@ -15,6 +15,7 @@ import GHC.Internal.Types () ( getSystemEventManager , getSystemTimerManager , ensureIOManagerIsRunning + , interruptIOManager , ioManagerCapabilitiesChanged , threadWaitRead , threadWaitWrite @@ -459,4 +460,8 @@ foreign import ccall unsafe "setTimerManagerControlFd" c_setTimerManagerControlFd :: CInt -> IO () #endif +-- No-op. Here to provide a uniform API to call in GHC.Internal.Conc.IO +interruptIOManager :: IO () +interruptIOManager = return () + #endif View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/4e7cb00e60c20eb041c8403dcf577bd… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/4e7cb00e60c20eb041c8403dcf577bd… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0

[Git][ghc/ghc][wip/dcoutts/io-manager-feature-bitset] 77 commits: [CLC 341] Strict variants of mutable types
by Duncan Coutts (＠dcoutts) 29 Oct '26

29 Oct '26

Duncan Coutts pushed to branch wip/dcoutts/io-manager-feature-bitset at Glasgow Haskell Compiler / GHC Commits: f8cfb1a5 by Hécate Kleidukos at 2026-10-04T19:36:39+02:00 [CLC 341] Strict variants of mutable types This commit is the implementation of CLC Proposal 341 https://github.com/haskell/core-libraries-committee/issues/341 - - - - - 575c4bc6 by Rodrigo Mesquita at 2026-10-05T12:32:02-04:00 ttg: HsModuleName in the AST, ModuleName in GHC This commit makes the AST fully independent from GHC by parsing a "module name" as an `HsModuleName (= HText)` rather than constructing an `ModuleName (= FastString)` straight away. In the GHC side, everything remains as a `ModuleName`. We turn AST `HsModuleName`s into fast strings with the cheap `mkFastStringShortText` when we process the AST for manipulation within the compiler. There are no performance regressions introduced by this approach. This is the final piece of #21592 Alternative plans were discussed in gitlab.haskell.org/ghc/ghc/-/merge_requests/16546#note_694647, but the Simon suggested one won in simplicity and performance when compared against my attempt at implementing my original plan of having a family `ModuleNameP`. - - - - - b3b89d32 by mangoiv at 2026-10-05T22:03:32-04:00 hadrian: Persist the used flavour name in the build root (#20650) This executes the idea proposed in #20650, teaching Hadrian to persist the flavour used to build a build root. This both makes use of multiple build roots more ergonomic (as the user no longer needs to repeat themselves) and avoids a tiresome foot-gun (forgetting to specify a flavour, often wholly invalidating a build root). Closes #20650. Co-authored-by: Sylvain Henry <sylvain(a)haskus.fr> - - - - - f977837b by mangoiv at 2026-10-05T22:04:21-04:00 configure: fix sed script and subst tooldir script Fixes #27672 Adds sanity checking for SUBST_TOOLDIR Metric Decrease: LinkableUsage01 MultiLayerModulesTH_OneShot T10547 T12234 T12425 T13035 T13820 T18140 T18730 T18923 T9198 TcPlugin_RewritePerf mhu-perf - - - - - f1cb08be by Hannes Siebenhandl at 2026-10-05T22:05:03-04:00 Prepend in-memory unit database for backpack The packageDbFlags are stored in reverse order. Thus, prepending will ensure that the in-memory unit database is the last item on the unit database stack. Further, the in-memory unit database needs to be able to shadow units from earlier unit databases in the unit database stack. Moreover, a `-clear-package-db` would remove the in-memory database if it weren't the last item on the stack. Closes #27835 - - - - - 8d49ed42 by Kazu Yamamoto at 2026-10-06T19:37:25-04:00 Do not leave standard file descriptors free at startup (#24450) A process may be started with stdout closed -- `prog >&-` from a shell does exactly that. Nothing then reserves descriptor 1, and open(), pipe() and socket() hand out the lowest free descriptor, so the next descriptor the runtime creates for itself takes it: the ticker's pipe, or a capability's IO manager control pipe. 'GHC.Internal.IO.StdHandles.stdout' is a CAF, built the first time the program writes to stdout, which is long after that has happened. It builds a 'Handle' over descriptor 1, which now belongs to the runtime, and nothing reports an error because the descriptor is perfectly good. The program then writes its output into the runtime's own plumbing. What that looks like from the outside is a hang. The descriptor the Handle has taken over is the read end of a pipe, so poll() for POLLOUT on it never says yes and epoll_ctl() accepts a registration for it; the thread waiting to write blocks for ever. Closing the Handle at exit closes the runtime's descriptor, which is worse: the ticker then fails on every tick. 'hs_init_ghc' now opens /dev/null onto whichever of descriptors 0, 1 and 2 are closed, before the runtime creates any descriptor of its own. Each is opened for the access that descriptor would normally have, read-only for 0 and write-only for 1 and 2, so that writing to 'stdin' or reading from 'stdout' still fails rather than quietly succeeding against /dev/null. See Note [Standard file descriptors must be open]. Moving only the descriptors the runtime opens would not do: the first file the program itself opened would land on the free number instead. The fix has to leave no free number. A program run this way therefore finds stdout connected to /dev/null rather than failing with EBADF on the first write, which is what such a program would get from most other language runtimes. Co-Authored-By: Claude Opus 5 (1M context) <noreply(a)anthropic.com> - - - - - 65bcf9e9 by Simon Jakobi at 2026-10-06T19:38:09-04:00 changelog: Repair the sentence describing #27477 The description read "programs using could crash", the object of "using" having been lost. The synopsis already says which programs are affected, so drop the dangling word. Assisted-by: Claude Fable 5.1 - - - - - 15ece296 by mangoiv at 2026-10-08T15:14:56-04:00 ci: move hadrian/ghci into the lint category so it can be picked up by more runners Fixes #27817 - - - - - c0823135 by mangoiv at 2026-10-08T15:14:56-04:00 ci: make lint-adjacent jobs into platform independent - move lint jobs into lint-anyarch tag - unify lint jobs under the same platform independent container - - - - - f76464cf by Zubin Duggal at 2026-10-08T15:15:39-04:00 DmdAnal: Make AbsDmd neutral in maxDmd fixes #27845 - - - - - ffa56ae4 by sheaf at 2026-10-09T17:15:59+02:00 Preserve sign of zero when parsing -0.0# This commit leverages the recently added GHC.Types.Literal.Floating module in order to preserve the sign of negative zero from source-written floating-point literals. Fixes #27545 - - - - - d538cac9 by Facundo Domínguez at 2026-10-09T20:26:38-04:00 Propagate annotations to simple interfaces This allows annotations to be visible to plugins when run by haddock. The LiquidHaskell plugin needs the annotations, in particular. Fixes #26761 - - - - - 32c4f4ab by meooow25 at 2026-10-09T20:27:28-04:00 Bump containers submodule to 0.8.1 And fix affected tests. * T4086a: containers depends on template-haskell-lift now instead of template-haskell. * T22611: Updated to be less dependent on containers internals. - - - - - c786065e by sheaf at 2026-10-09T20:28:20-04:00 Use an explicit pattern match in toIfaceIdDetails The old wildcard pattern let this code bit rot: it did not properly account for recently added 'IdDetails' such as 'RepPolyId', which resulted in spurious 'pprTrace' messages that could even pollute HLS sessions. Fixes #23785 - - - - - e7cacf59 by sheaf at 2026-10-09T20:29:09-04:00 Take into account ifKind in freeNamesIfDecl Commit 3c2f4bb4 added the 'ifKind' field to various constructors of 'IfaceDecl', which preserves the original kind (before type synonym expansion) for better error messages. However, that commit forgot to update 'freeNamesIfDecl' correspondingly. This could result in a "urk! lookup local fingerpring" crash when e.g. a type's kind uses a type synonym defined later, as we failed to add a dependency on the type synonym during dependency analysis. Fixes #27916 - - - - - 41e80f41 by Simon Peyton Jones at 2026-10-11T14:38:51-04:00 Simplify the error-suppression mechanism in GHC.Tc.Errors In `GHC.Tc.Errors.reportWanteds` we suppress some errors in favour of others. But the mechanism had grown crufty, and #27331 exposed a bug. This MR makes it simpler and more uniform, by doing everything via the `ei_suppress` field of the ErrorItem. Claude then found another closely-related bug (see the review linked on !16564). That bug is immortalised in new test T18851d. I'm still not happy with `ignoreConstraint` but we can worry about that another day; I have not touched it. Main changes * Instead of /predicting/ when an error will be reported, we set `cec_error_reported` when an error is /actually/ reported. (I renamed it from `cec_suppress` to reflect this change.) Now errors are suppressed only when a preceding error has actually been reported. Simpler and more robust. See `Note [cec_error_reported: suppressing less-serious errors]` The big changes are in `reportWanteds` which looks very different. * New `Note [ei_suppress: suppressing confusing errors]` describes the (quite separate) `ei_suppress` mechanism for suppressing confusing errors, mainly arising from non-empty rewriter sets * In `tryTypeClassDefaulting`, instead of switching off type-class defaulting if there is an insoluble /anywhere/, we just switch it off locally when we find an insoluble sub-tree. See Note [When to do type-class defaulting] and (W5) in Note [ApproximateWC] * Greatly simplify `insolubleWantedCt`. * I made type-class-defaulting be a bit more eager, only switched off when a peer or ancestor (but not child) constraint is insoluble. See Note [When to do type-class defaulting]. Small refactorings * Rename `askNoErrs` to `askErrsFound` and flip its polarity * The code for "check if ei_suppress is true of all items" code is moved to `mkErrorItems` rather than in the large `reportWanteds`. * Code for `has_gadt_match` moves to `mkGivenErrorReporter`, rather than polluting the large `reportWanteds`. * Many tests with `-fdefer-type-errors` get more warnings, especially about ambiguous (Num a) constraints. This is by-design: see Note [Always warn with -fdefer-type-errors]. Previously GHC was not honouring this Note but now it is, hence the extra errors. - - - - - f217fc31 by Cheng Shao at 2026-10-11T14:39:33-04:00 compiler: ensure cross ghc --info output target unit ids Previously, for cross ghcs, the base/ghc/ghc-internal unit ids in --info output do not match the target unit ids reported by ghc-pkg, breaking cross ghc release builds with hash-unit-ids when being used with cabal 3.18. This patch fixes it by making --info output target unit ids. Fixes #27919. Co-authored-by: Codex <codex(a)openai.com> - - - - - d07a5176 by Simon Peyton Jones at 2026-10-13T18:10:39-04:00 Remove the Eq and Ord instances for ForAllTyFlag In Note [Comparing visibility] in GHC.Core.TyCo.Compare we read that "we always compare ForAllTyFlags with `eqForAllVis` or `cmpForAllVis`" But alas there was an Eq and Ord instance for ForAllTyFlag, which made it impossible to check this claim. So I made these changes: * Move the definition of ForAllTyFlag and its related functions from L.H.S.Specificity (where it did not belong, since L.H.S didn't use it) into GHC.Types.Var. * Remove the Eq and Ord instances for ForAllTyFlag * Deal with the fallout, by using `eqForAllVis` or related functions. * In interface files we use IfaceType as a key for a de-deduplication map See Note [Deduplication during iface binary serialisation] in GHC.Iface.Binary. For this we need /exact/ comparison, so we use a newtype wrapper for IfaceForAllTyFlag, and give use `cmpForAllTyFlagExactly` to implement its Ord instance. Tiresome but easy. * Rename `toIfaceForAllBndrs` to `toIfaceForAllTyBndrs` - - - - - ea5c054d by Simon Peyton Jones at 2026-10-13T18:10:39-04:00 Refactor type and coercion substitution This MR refactors type and coercion substitution so that they use TyCoMapper, rather than hand-rolled code. Fundamentally it's a simple idea, but I had to do quite a bit of refactoring to make it fly. I got Claude to help me this this refactoring, but I paid careful attention to every line of code. Main changes: * New module GHC.Core.TyCo.Make which defines TyCoMapper, and a bunch of other simple "smart constructors" for types and coercions. See GHC.Core.TyCo.Rep Note [The Type-related module hierarchy], esp (TRM1) * The new TyCoMapper has new fields `tcm_tcapp_ty` and `tcm_tcapp_co` to handle type-constructor applications. These replace `tcm_tycon`. * Rewrite GHC.Core.TyCo.Subst to use TyCoMapper * Rewrite GHC.Core.Type.expandTypeSynonyms to use TyCoMapper. * Implement `seqType` and `seqCo` with TyCoFolder. To to this I had to add a `tcf_lit` field to TyCoFolder. * A new module GHC.Utils.StrictIdentity to define a strict identity monad, which itself is used for substTyCoMapper :: TyCoMapper Subst StrictIdentity * I removed an apparently-unused instance for Binary Levity Refactoring `seq` functions * In GHC.Core.Seq, redefine `seqOneShot`, `seqRuleInfo`, and friends to return `SeqResult`, a new type defined in GHC.Types.Baseic, rather than `()`. We can define a Monoid instance for `SeqResult` that does the right thing. * This has knock-on effects across the compiler. - - - - - 34165e23 by Simon Peyton Jones at 2026-10-13T18:10:39-04:00 Refactor TyCoMapper to take a single monad; add StrictReaderT TyCoMapper used to be parameterised over both an environment and a monad, with fields like tcm_tyvar :: env -> TyVar -> m Type Now it is parameterised over just the monad: tcm_tyvar :: TyVar -> m Type If the mapping needs an environment, `m` is a reader monad over that environment. Many uses (e.g. zonkTcTypeMapper, swizzleMapper) need no environment at all, and used to pass a dummy (). The binder field becomes tcm_tycobinder :: forall r. TyCoVar -> ForAllTyFlag -> (TyCoVar -> m r) -> m r and runs the continuation in the extended environment. mapTyCoX is gone; only mapTyCo remains. To make this efficient, the reader monad must use the one-shot trick and be strict in the environment. So: * New module GHC.Utils.Monad.StrictReader defines newtype StrictReaderT r m a with hand-written instances. The builder of the pattern synonym `StrictReaderT` is a smart constructor that adds both `oneShot` and a bang on the environment, so every StrictReaderT computation is strict in the environment. See Note [Instances for StrictReaderT]. * ZonkT becomes type ZonkT = StrictReaderT ZonkEnv ZonkM becomes type ZonkM = StrictReaderT ZonkGblEnv IO replacing their own hand-written instances. * Substitution uses type SubstM = StrictReaderT Subst StrictIdentity substTyCoMapper :: TyCoMapper SubstM and similarly expandTypeSynonymMapper. * zonk_tycomapper :: TyCoMapper (ZonkT TcM) is now much simpler. * swizzleMapper uses StrictIdentity rather than Identity. * Module renaming, so that the strict monads are all peers in GHC.Utils.Monad: GHC.Utils.StrictIdentity ==> GHC.Utils.Monad.StrictIdentity GHC.Utils.Monad.State.Strict ==> GHC.Utils.Monad.StrictState alongside the new GHC.Utils.Monad.StrictReader. Co-Authored-By: Claude Opus 5.5 <noreply(a)anthropic.com> - - - - - c0299e15 by Simon Jakobi at 2026-10-13T18:11:23-04:00 testsuite: Line-buffer the driver's stdout Python block-buffers stdout when it is a pipe. This is the case for example in CI. As a result there is often no indication of progress at all until the run ends. To fix this, we configure it explicitly to use line buffering instead. stderr is reconfigured too, for consistency: Python line-buffers it by default since 3.9. The type-ignores are for mypy, which types these handles as TextIO — an interface without reconfigure. Fixes #12934. Assisted-by: Claude Fable 5 - - - - - 9b5beaa2 by Florian Ragwitz at 2026-10-13T18:12:05-04:00 hpc: Improve coverage semantics for instance methods and instances Users care about which of their explicitly written methods are or aren't covered, not whether all of the inherited methods are covered as well. Those should be covered where they are implemented. We therefore skip generating top-level boxes for inherited methods which will appear as uncovered with no clear indication as to why, and no way to fix other than exercising code the user has not written. Previously, top-level boxes for those inherited methods would use the instance head as their source span. Now that those boxes are gone, we can use that source span for another purpose: We introduce a new top-level box for each class instance with its instance head as the source span, and use that to indicate whether or not an instance has been covered at all. We do this by making any method of the instance, inherited/generated or otherwise, tick the box for the instance head. Excluded from this are methodless classes, which could never be covered using the current mechanism. This could be improved upon further. These changes apply to ticking for program coverage only, and don't affect source notes. Additionally, we update the scope of record selector ticking to be limited to program coverage as well, as the oversight to also extend it to source notes has previously caused bugs and because record selector ticking is coverage-specific. Fixes #17155, #15932. - - - - - 836ec50a by Simon Peyton Jones at 2026-10-13T18:12:54-04:00 Refactor fundeps again This MR fixes #27828; there was a bad scoping error in `trim_qtvs`. The main changes are * Refactor `trim_qtvs` completely to eliminate the bug. * Remove a redundant `apartnessCheck` from `mkTopOpenFamEqFDs`. * Lots of improvements to the Notes in FunDeps.hs I also did a couple of pure refactors: * I realised that we were carefully caching `cab_incomps` in each `CoAxBranch`, to make it more efficient to reduce a type family application, but we were not doing the same thing when computing relevance for functional dependencies; see Note [Relevance] in GHC.Tc.Solver.FunDeps. So I added `cab_overlaps`, which is now used in `noEarlierMatch`. * Use a proper data types, `AmIUnifying` and `OuterKindsFlag`, for two previously-Boolean arguments to `tc_unify_tys`. This is a pure refactor. - - - - - 889ef371 by Simon Peyton Jones at 2026-10-13T18:13:36-04:00 Refactor and simplify solver's unification code Ticket #27844 showed that GHC's constraint solver could miss an occurs check, and thereby build an infinite type. This patch significantly simplifes the code in GHC.Tc.Solver.Equality.canEqCanLHSFinish_try_unification GHC.Tc.Solver.Equality.canEqCanLHSFinish_no_unification It is now significantly simpler, and requires much less subtle reasoning to understand. Main changes * If `canEqCanLHSFinish_try_unification` fails to unify, it now /always/ calls `canEqCanLHSFinish_no_unification` rather than trying to finish the job itself. * `GHC.Tc.Solver.Monad.checkTypeEq` is moved from Monad into Equality, and renamed to `GHC.Tc.Solver.Equality.checkTypeEqNoUnification`. * `canEqCanLHSFinish_no_unification` calls `checkTypeEqNoUnification` to test the invariants for an inert equality. Before this MR there was a special case for `do_not_prevent_rewriting`, but that has gone; instead it's just a consequence of the flags passed to `checkTyEqRhs`, documented in the `PuOK` case of `canEqCanLHSFinish_no_unification`. * There is only one bit of "swap back" code, in `finish_no_unify` in `canEqCanLHSFinish_try_unification. Previously this code was duplicated. * `kickOutAfterFillingCoercionHole` should account for equalities with a TyFamLHS. See (KOC5) in Note [Kick out after filling a coercion hole] * No need to gather coercion holes in the `PuOK` case of `canEqCanLHSFinish_no_unification`. See documentation with that code. - - - - - f9193e53 by Simon Peyton Jones at 2026-10-13T18:13:37-04:00 One line fix to fallout from ...#27731, !16564, simplifying Tc.Errors machinery There was an over-eager assert! - - - - - cdd56b7a by Florian Ragwitz at 2026-10-13T18:14:19-04:00 Fix several sphinx errors and warnings from rendering the release notes The extension references which are removed here are for extension which were already previously undocumented, so just removing them seems fine for now. - - - - - 4f00da25 by Florian Ragwitz at 2026-10-13T18:14:19-04:00 Link to QualifiedStrings documentation - - - - - 97adaf89 by Simon Peyton Jones at 2026-10-14T00:07:27+01:00 Line up simpleUnifyCheck and check_ty_eq_rhs Even after #26543 was allegedly fixed, the original repo case in the Description continued to fail. The reason was that the QuickLook unifier, `qlUnify`, used `simpleUnifyCheck` for checking unification invariants; and `simpleUnifyCheck` conservatively rejected any RHS with a coercion hole in it. The upshot was that QuickLook was not as clever as it should be, wrongly failing to do an impredicative instantiation. Interestingly `check_ty_eq_rhs`, which does the same job during constraint solving, was more liberal: it just looked at the free vars of the coercion, and allowed coercion holes. This MR lines them up, adding some careful notes. See Note [simpleUnifyCheck] Note [Occurs check and coercions] Note [Unification preconditions] esp (COERCIONS) Things are better than before, but I am still uncomfortable about the possibilty that a hole might be filled with a coercion that mentions the LHS type variable; for now I have left this discomfort documented in Note [Occurs check and coercions]. Main changes * `qlUnify` now only unifies /instantiation variables/ not regular unification variables. This makes QuickLook more predictable. See (UQL6) in Note [QuickLook unification]. * `qlUnify` no longer needs to call `simpleUnifyCheck`, so we can get rid of `UC_QuickLook` and its code in `simpleUnifyCheck`. * I found I could get rid of `mkOccFolders` in favour of `anyFreeVarsOfType`. Hooray. * Improve documentation of CycleBreakerTv, and rename it to GivenCycleBreakerTv because they are only uesd for Givens! * I accepted a 9% worsening in compile time allocation on T5030 because the newly-consistent story triggered #27934. Other changes * I refactored `atvFolder` so that it pairs the "interesting-var" function with the in-scope set, to avoid building a closure with a free variable. Now we get nice tight loops for `anyFreeVarsOfType` and friends. See Note [Any-free-var folder]. Metric Increase: T5030 - - - - - 5f7b3047 by Simon Peyton Jones at 2026-10-14T00:07:27+01:00 Fix order-dependence in QuickLook Ticket #27917 pointed out that QuickLook was order-dependent -- very bad! The solution is in `qlUnify`: spot when we are unifying a type with a definite monotype, and then monomorphise any free instantiation variables of that type. See (UQL8) and (UQL9) in Note [QuickLook unification]. - - - - - c391c70c by Simon Peyton Jones at 2026-10-14T00:07:28+01:00 Add some comments about RuntimeUnk This commit just adds comments explaining why we don't need to worry about RuntimeUnkTv in the constraint solver. It also tightens up docmentation of earlier (overlapping) commits in this MR. - - - - - 7df3ba2f by Andrea Vezzosi at 2026-10-14T06:15:48-04:00 add runGhcWithSignalHandlers and remove handling from runGhc[T] it only ever makes sense to call `withSignalHandlers` from `main`, otherwise signals can be too easily swallowed without notification to the rest of your process. This commit takes withSignalHandlers out of `runGhc/runGhcT` and provides `runGhcWithSignalHandlers` for the old behaviour. - - - - - 3a72b8a2 by Hannes Siebenhandl at 2026-10-14T13:48:32-04:00 Fix #27836, normalise unit database path in ExternalUnitDatabases Add a regression test, proofing that all variations: - local-unit-db/my-test.conf - ./local-unit-db/my-test.conf - ./local-unit-db/my-test.conf/ are parsed only once. - - - - - 8ddbc6f9 by Florian Ragwitz at 2026-10-14T13:49:28-04:00 Remove obsolete HPC types Fixes #27946. - - - - - f34c79ee by Alan Zimmerman at 2026-10-15T04:54:47-04:00 EPA: Start using vocurly as basis for AnnListLayout For the existing cases that populate AnnList. This uses AnnListBraces when braces are explicitly supplied, and AnnListLayout when vocurly is used The change also shows up AnnList usages that can never capture layout, these are cleared up later. - - - - - 033ba59c by sheaf at 2026-10-15T04:55:44-04:00 Make exprFreeVars deep exprFreeVars used to be a shallow free-variable finder (it didn't look into kinds), but most callers wanted it to be deep. In particular, 'exprFreeVars' is often used to seed an 'InScopeSet', and that must contain the /deep/ free variables. This commit makes it deep, which allows us to get rid of calls to 'closeOverKinds' or other workarounds for shallowness in: - 'GHC.Core.Opt.Simplify.Utils.abstractFloats' - 'GHC.HsToCore.Binds.dsSpec_help' - 'GHC.Core.Opt.Arity.etaBodyForJoinPoint' - 'GHC.Core.FVs.freeVars' - 'GHC.Core.Lint.lintCoreRule' We also remove 'deepExprsFreeVarsDSet' and 'closeOverKindsDSet'. Other places needed a little bit more care, in particular to properly handle coercion variables free in kinds: - 'GHC.Core.Opt.Specialise.specHeader' quantifies over the binder of an unspecialised argument directly, instead of computing the free Ids of the argument. - 'GHC.HsToCore.dsWarnRuleShadowing' filters out 'CoVar's. - 'GHC.Core.Opt.SpecConstr.callToPat' had some ad-hoc logic to compute argument pattern types and a workaround with 'expandTypeSynonyms'. That logic is removed in favour of a new 'argsToPats' function which directly computes the argument pattern types by instantiating the type of the head of the call pattern. Fixes #27936 - - - - - f93d523c by Luite Stegeman at 2026-10-15T04:56:34-04:00 testsuite: Use mkWeakMVar in memo001/memo002 mkWeakPtr in the memo001/memo001 tests created weak pointers with the MVar box as the key, which is fragile in the presence of optimization. The fix is using mkWeakMVar, keying on the primitive MVar# instead. Fixes #27924 and #27396 - - - - - 8c947573 by Simon Peyton Jones at 2026-10-16T13:11:53+01:00 Improve exprOkForSpeculation Fixes #27910, which exposed an infelicity in exprOkForSpeculation The new story is carefully described in new Note [exprOkForSpeculation], esp (a-c) and (EOKS1). - - - - - 7dedcf13 by Alan Zimmerman at 2026-10-16T19:03:17-04:00 EPA: Add TTG fields to FamilyInfo This commit is a simple add, with unused extension fields. It lays the groundwork for using them for exact print annotations - - - - - 26b9a8a0 by Arkissa at 2026-10-17T16:04:10+00:00 ghci: Prefer exact source spans in :type-at Prefer expression types for identical source spans. Fixes #27412. - - - - - 70dc4e61 by sheaf at 2026-10-18T18:15:15-04:00 Carefully implement coercion zapping This commit implements coercion zapping in the rewriter, controlled by the -fzap-coercions flag (on by default). This significantly reduces the size of coercions produced when rewriting type family applications, resulting in much faster compilation. See Note [Coercion zapping] in GHC.Tc.Solver.Rewrite for an explanation and further details. The rewriter now keeps track of the `CoVar`s and `CoercionHole`s it uses when rewriting, which allows computing the dependencies of a zapped coercion without traversing the coercion the rewriter produced, as explained in Note [Computing coercion dependencies] in GHC.Tc.Solver.Rewrite. Accompanying changes: - Rework how dependencies of 'UnivCo's are tracked, purely for performance reasons. We now use the 'UnivCoDeps' datatype instead of '[Coercion]', and we now insist that 'uco_deps' also contains the CoVars that appear in the LHS and RHS types of the UnivCo (as a caching mechanism). - Several performance improvements to the rewriter (e.g. increased strictness). - Improvements to the family application cache, including: - Better documentation. - Only cache a whole chain of reduction steps. - Don't put coercion holes in the cache, fixing #27518. - Allow Core Lint to be called from within the typechecker (and from within the coercion optimiser). - Move out "annotation linting" from Core Lint into the general Core pipeline, to avoid the dependency GHC.Core.Coercion.Opt->GHC.Core.Lint causing a regression in CountDepsParser due to transitive dependencies of GHC.Core.Opt.Monad and GHC.Unit.Module.ModGuts. Fixes #8095 #27336 #27518 Summary of performance numbers Compile-time byte allocated in GHC's testsuite (0 regressions) CoOpt_Singletons(normal) ghc/alloc 634,040,332 595,448,728 -6.1% GOOD FamAppCachePerf(normal) ghc/alloc 495,629,812 191,612,920 -61.3% GOOD LargeRecord(normal) ghc/alloc 1,226,806,434 966,260,976 -21.2% GOOD SimplCastPerf(normal) ghc/alloc 160,505,678 148,112,688 -7.7% GOOD T12227(normal) ghc/alloc 244,668,982 233,037,216 -4.8% GOOD T12545(normal) ghc/alloc 697,804,842 424,705,152 -39.1% GOOD T13386(normal) ghc/alloc 422,871,076 35,213,656 -91.7% GOOD T15703(normal) ghc/alloc 288,402,364 273,692,560 -5.1% GOOD T26426(normal) ghc/alloc 86,002,052 78,152,104 -9.1% GOOD T27336(normal) ghc/alloc 498,932,268 35,886,016 -92.8% GOOD T5030(normal) ghc/alloc 117,057,706 88,785,648 -24.2% GOOD T5321Fun(normal) ghc/alloc 252,552,628 243,820,176 -3.5% GOOD T8095(normal) ghc/alloc 768,198,564 127,779,520 -83.4% GOOD T9872a(normal) ghc/alloc 1,655,675,020 1,406,017,008 -15.1% GOOD T9872b(normal) ghc/alloc 1,864,180,628 1,644,706,696 -11.8% GOOD T9872b_defer(normal) ghc/alloc 2,834,585,468 1,657,729,760 -41.5% GOOD T9872c(normal) ghc/alloc 1,572,573,116 1,354,055,424 -13.9% GOOD T9872d(normal) ghc/alloc 351,516,004 331,071,424 -5.8% GOOD TcPlugin_RewritePerf(normal) ghc/alloc 2,063,065,680 1,847,464,272 -10.5% GOOD Chesskell (github.com/Proxymoron461/chesskell/issues) Typechecking that the first N plies of kholmovVsBronsteinShort are a valid chess game: Plies | GHC 9.14 time/alloc | This commit time/alloc | -----:|---------------------|------------------------| 2 | 41s/20GB | 0.2s/19MB | 4 | 95s/49GB | 0.3s/25MB | 6 | timed out | 0.6s/522MB | 8 | timed out | 7s/673MB | 10 | timed out | timed out | Compilation of singletons-base: compile time -28%, allocations -25% ------------------------- Metric Decrease: CoOpt_Singletons FamAppCachePerf LargeRecord SimplCastPerf T12227 T12545 T13386 T15703 T24984 T26426 T27336 T5030 T5321Fun T8095 T9872a T9872b T9872b_defer T9872c T9872d TcPlugin_RewritePerf ------------------------- Co-Authored-By: Sylvain Henry <sylvain(a)haskus.fr> - - - - - 3d326d7d by Sylvain Henry at 2026-10-18T18:16:09-04:00 JS: fix stack corruption in h$ap_gen_fast's pushAllRegs (#27624) The loop added in 9e46990976 ("JS: support more than 128 registers (#26558)") pushed register N to stack[sp - N - 2] while the switch it falls through into, and every reader of the h$ap_gen frame, uses stack[sp - (N - 2)]. Registers >= R32 were therefore written 4 slots too low: the slots the readers consult were left stale and the frame below was silently corrupted. Fix the loop to use the same slot formula as the switch, and add a regression test: an unknown call to a thunk with 35 argument register slots, which goes through the generic apply. Co-Authored-By: Claude Fable 5 <noreply(a)anthropic.com> - - - - - 6bbca569 by Andreas Klebinger at 2026-10-21T10:01:10+00:00 Fix a few tag inference bugs. * Treat bottoming functions and nullary join points as functions. * When generating bytecode don't rely on imported tag sigs. Fixes #27831, #27841 and #27842. - - - - - 71808e29 by Sylvain Henry at 2026-10-22T04:50:24-04:00 Add GHC.Internal.Unboxed.Do: do-notation for unboxed values (#27944) QualifiedDo operators can be representation-polymorphic: a do-block is expanded into ordinary applications before typechecking, and the only check on a bound variable is the deferred fixed-runtime-rep check. A representation-polymorphic (>>=) cannot be defined directly, though, because its body would bind a variable of unknown representation. The new module provides such operators for State#-passing code as methods of a class indexed by RuntimeRep, with one instance per representation. With optimisation the dictionaries disappear and a block compiles to the usual nested case expressions: Ubx.do { x <- readWordArray# mba 0# -- x :: Word# ; writeWordArray# mba 1# x -- unit actions are statements ; Ubx.pure x } Also add users guide documentation and tests. Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com> - - - - - 33247780 by Sylvain Henry at 2026-10-22T04:50:24-04:00 Bignum: use Ubx.do for State#-passing code (#27944) Rewrite the State#-passing code of the GHC.Internal.Bignum modules (Primitives, WordArray, BigNat, Natural, Integer, Backend.Native, Backend.GMP) with the qualified do-notation of GHC.Internal.Unboxed.Do. Nested case chains such as case newWordArray# sz s of { (# s, mwa #) -> case act mwa s of { s -> unsafeFreezeByteArray# mwa s }} become Ubx.do mwa <- newWordArray# sz act mwa unsafeFreezeByteArray# mwa The optimised Core is unchanged (compared per top-level binding). BigNat now enables MonoLocalBinds: local loops written with Ubx.do must stay monomorphic in the state type, otherwise GHC does not turn them into join points. Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com> - - - - - 593f5f5b by Sylvain Henry at 2026-10-22T04:51:16-04:00 JS: remove archive padding after wasm objects (#25665) On Darwin hosts, emar (llvm-ar) writes archives in the Darwin format when the first member is not a known object file, which is the case for the archives of the JS backend. This format pads each member to 8 bytes with '\n' and includes the padding in the member size. Wasm objects do not accept trailing bytes: wasm-ld failed with "section too large". The JS linker now removes this padding when it extracts a wasm object from an archive. In addition, Hadrian passes --format=gnu to emar for the JS target. See Note [Wasm objects in Darwin archives]. Also document the format of Cc (wasm) objects in Note [Cc object format]. Co-Authored-By: Claude Opus 5.5 <noreply(a)anthropic.com> - - - - - 13dff5d0 by L0neGamer at 2026-10-22T21:37:31-04:00 add options for resizing bytearrays (#27244) add five mutable bytearray resizing operations for various kinds of pinnedness and aligning. - prefer unpinned when allocating new array (``resizeMutableByteArrayPreferUnpinned#``; the ``resizeMutableByteArray#`` default) - prefer pinned when allocating new array (``resizeMutableByteArrayPreferPinned#``) - prefer aligned and pinned when allocating new array (``resizeMutableByteArrayPreferAlignedPinned#``) - prefer same as input bytearray when allocating new array (``resizeMutableByteArrayKeepPinnedness#``) - ensure the output array is aligned and pinned after operation (``resizeMutableByteArrayMakeAlignedPinned#``) - - - - - 0c17201c by Sylvain Henry at 2026-10-22T21:38:23-04:00 JS: implement performMinorGC and performBlockingMajorGC (#28012) GHC.Internal.System.Mem imports three C entry points: performMajorGC, performGC (for performMinorGC) and performBlockingMajorGC. The JS RTS only defined h$performMajorGC, so calling performMinorGC or performBlockingMajorGC threw "ReferenceError: h$performGC is not defined" at runtime. The JS RTS has a single non-generational collector, so both missing entry points are now aliases of h$performMajorGC. Add a test that calls every GC entry point of System.Mem. Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com> - - - - - 7a631ff2 by L0neGamer at 2026-10-23T09:11:39-04:00 Mon(a|oi)d of No (Return|Mappend) progress (!15795) Make non-canonical definitions of return and mappend non-suppressible errors, and make canonical definitions of those methods warnings. CLC proposal: https://github.com/haskell/core-libraries-committee/issues/328 - - - - - 14656fd4 by Simon Jakobi at 2026-10-23T09:12:27-04:00 Cmm: keep the CBE substitution idempotent When blocks are merged over multiple rounds during common block elimination, a block that won a merge in an earlier round can still lose a merge in a later round. The substitution map (Subst) used to record this as a chain: the earlier loser mapped to a block that was itself eliminated. In PLT terminology, the substitution became *non-idempotent*. Applying such a substitution with a single lookup left edges to unreachable blocks in the Cmm graph, eventually leading to a setInfoTableStackMap panic (#27368). With this patch, each round's fresh substitutions are applied to the old ones as they are added (extendSubst), so every eliminated block maps directly to its final replacement and the substitution stays idempotent. Fixes #27368. Assisted-by: Claude Fable 5.1 - - - - - 92848e61 by Evgeny Malyshev at 2026-10-23T20:26:04-04:00 hadrian: Include transitive dependencies in Haddock builds Haddock needs indirect dependency interfaces to resolve references to definitions outside a package's direct dependencies. Expose Cabal's configured dependency closure through ContextData and use it alongside the direct list, preserving direct-interface precedence. Bump Shake's cache version for the serialized oracle change. Add a Hadrian selftest requiring template-haskell's indirect ghc-prim interface. Fixes #25850. Fixes #27786. Assisted-By: OpenAI Codex Metric Increase: haddock.Cabal - - - - - 5e8ae81b by Zubin Duggal at 2026-10-23T20:26:47-04:00 WinIO: Don't confuse Win32 errors and NTSTATUS codes WinIO misinterpreted NTSTATUS codes as Win32 errors, and had some ad hoc logic to deal with this. However, this logic was incomplete and resulted in the mishandling of results of certain I/O operations (see testcase). Now the `ErrCode` argument in `CompletionCallback` is always the Win32 error code, and we convert the NTSTATUS using RtlNtStatusToDosError. Also when an overlapped/async operation such as ReadFile fails immediately, with a Win32 error such as ERROR_BROKEN_PIPE, we don't get a further completion notification, so we never recorded the failure. This worked by coincidence earlier, because `STATUS_PENDING` happened to have the same numerical code as `ERROR_NO_MORE_ITEMS`, but it doesn't now that we translate `STATUS_PENDING` into the correct Win32 error code (`ERROR_IO_PENDING`). Fix this by using the Win32 error value we obtained when an operation returns `STATUS_PENDING`. Fixes #27971 - - - - - c42ed66a by Martijn Bastiaan at 2026-10-24T16:00:42+02:00 Add single-threaded test for #27526 + #27123 Issue #27526 is a duplicate of #27123, which has already been fixed in HEAD. The proposed patch in the former carries a "nicer" regression test in that it doesn't rely on multi-threading to trigger. Closes #27526 - - - - - 7f73f515 by Simon Peyton Jones at 2026-10-24T17:29:19-04:00 Refactor QuickLook a bit This MR simply refactors QuickLook a bit. In particular, it simplifies `tcApp`, and moves shared code into `finishApp`. It also adds `Note [QuickLook: arguments before result]` responding to #26599. There should be no user-visible changes, except perhaps in error messages, which improve marginally. For some reason T24984 compiles 8% faster. I'll take the win! Metric Decrease: T24984 - - - - - a4a3a964 by Alan Zimmerman at 2026-10-24T17:30:03-04:00 EPA: Use AnnList for XClosedTypeFamily We now have TTG annotations for FamilyInfo. Make use of it to store the AnnList annotations for a closed type family. - - - - - 953e4c7f by sheaf at 2026-10-26T18:19:10-04:00 Rework LogQueue to be a proper queue Now 'LogQueue' uses a 'TQueue' with a separate 'TVar' to indicate whether the log queue is finished logging, instead of the previous weird data representation using a reverse-order '[Maybe Msg]' with 'Nothing' denoting the end of the queue. - - - - - ec29c5fa by Rodrigo Mesquita at 2026-10-26T18:19:10-04:00 Refactor GHC.Driver.MakeAction Pull out of 'runParPipelines' the logic for creating a 'MakeEnv' ready to be used by multiple threads, as that will be useful for downsweep as well (which doesn't fit the 'runPipelines' flow), rather than being just for upsweep. The code is moved and re-structured to match the export list, simplify the sequentiality checks previously both in 'runPipelines' and 'runAllPipelines', which were weirdly similar and confusing; into the two part step where we need these checks: (1) to construct the MakeEnv, (2) to run the MakeActions in parallel. These two steps are separate and used to be too mixed up. Some additional little simplifications or clean ups here and there. - - - - - 3fb22073 by Rodrigo Mesquita at 2026-10-26T18:19:11-04:00 Refactor GHC.Driver.Downsweep in preparation for parallel downsweep Pure refactor to improve the code to facilitate implementing parallel downsweep in the next commit. This commit puts a MakeEnv into the DownsweepEnv, gives the fields proper names and uses RecordWildcards to simplify, rather than passing around all diagnostic wrappers, driver-message-things and using 10s of positional fields. No behavior changes here! - - - - - 52f385a7 by sheaf at 2026-10-26T18:19:11-04:00 Driver: enableCodeGenForTH in load', not downsweep enableCodeGenForTH modifies the module graph for planning, creating fresh temporary paths. So it should only be run when we are about to execute a build plan (e.g. in load'), so that consumers that want to know facts about the module graph don't go looking for files that don't exist yet (in particular 'haddock --no-compilation'). - - - - - 82402d14 by sheaf at 2026-10-26T18:19:11-04:00 2-phase Cache/Search Finder monad This commit restructures the finder abstraction by introducing the 'FinderM' monad, which splits module lookup operations into two phases: - a cache-only phase, performing no filesystem access, - from the first cache miss onwards, a search action which may access the filesystem. This allows consumers to distinguish between quick cached results versus more expensive filesystem search operations. - - - - - 5c302b40 by sheaf at 2026-10-26T18:19:11-04:00 Separate Home/External finder caches The caches for home modules and modules from external units have different lifetimes, needing to be invalidated at different points. This commit moves the cache for modules from external units into the 'UnitState' type, which keeps the cache correct whenever package flags or package databases change (setSessionDynFlags, :set -package ..., Backpack's addUnit). This also makes it easier to clear the cache of home modules, as they are now stored separately (obviating the 'isUnitEnvInstalledModule' check we used to perform). - - - - - 605fbb96 by sheaf at 2026-10-26T18:19:11-04:00 Add known home modules to the finder cache The finder looks in the file system for the on-disk source that corresponds to a Haskell module name. There are two kinds of modules where we need to bypass this search: - a file passed directly in the command line, which may not be in any search path and whose module name may differ from the file name; - Backpack signatures and modules, which do not exist as files at all. This commit handles these two situations by adding an immutable 'KnownHomeModules' in the finder cache, which is always consulted first before doing any search. This removes the ad-hoc imperative logic which relied on mutating the finder cache at careful points. 'setModuleGraph' now also takes the known home modules, so that the module graph and the known home modules are always set consistently. See Note [Known home modules] in GHC.Unit.Finder.Types. - - - - - 82718f44 by sheaf at 2026-10-26T18:19:11-04:00 Make 'UnresolvedImport' into a suitable cache key The goal of this commit is to make 'UnresolvedImport' suitable for use as a cache key. Specifically, during downsweep we want to be able to cache module resolution queries. We want to do this using some kind of map, e.g. (roughly speaking): Map UnresolvedImport ResolvedImport For this to work, 'UnresolvedImport' must contain **precisely** the input data to each resolution computation. In other words: 1. If something changes that materially affects resolution, it must be stored in the cache key. 2. If changing some input does not materially affect the outcome, then it must not be stored in the cache key. To satisfy (1), we change 'UnitNode' to also keep track of the home unit in whose unit state the unit's dependencies were looked up. For (2), 'ImportResolution' now stores a 'ModuleLookupScope' instead of an 'UnresolvedImportOrigin'. Plugin imports are expressed with a new 'LookupPlugin' scope. The source location of an import, which is immaterial to the resolution, is stored out of band outside 'UnresolvedImport', in 'ModSummary'. - - - - - d4635c7c by sheaf at 2026-10-26T18:19:11-04:00 Add GHC.Data.Dependent This commit adds a minimal implementation of dependent sums ('Some', 'DSum') and dependent maps ('DMap'). The implementation is not entirely performance optimal, as there are a few extraneous allocations compared to the full-blown implementation in the 'dependent-map' package. For our use case (caching downsweep computations), this does not matter much. See GHC.Data.Dependent. - - - - - 6759533e by sheaf at 2026-10-26T18:19:11-04:00 Driver: structured concurrent worker abstraction This commit introduces a structured concurrency framework in the style of the 'ki' library: a collection of threads within a scope. We implement two kinds of concurrent workers on top of this framework: - Independent workers cannot wait for one another at all. The only scheduling operation is to wait for quiescence. - Coordinating workers declare an STM readiness condition (waiting on other workers to complete) which gates their start. See Note [Deterministic concurrent workers] in GHC.Driver.Concurrency. This commit ports upsweep to this new framework, with downsweep being left as subsequent work. Further changes along the way: - Refactoring of how concurrency is acquired to avoid the footgun of trying to use a no-op 'AbstractSem' as a lock in the serial case. - The "re-run with -j1" logic for semaphore opening errors no longer triggers on late semaphore failures (part-way through a lengthy computation). - Logger threads are properly cleaned up on exception, with each concurrent worker's log queue and local TmpFs properly bracketed. - The 'GhcMessage -> AnyGhcDiagnostic' and 'Maybe Messager' arguments of 'depanalE', 'depanalPartial' and 'downsweep', which were all dead in practice, have been dropped. - Concurrent logging has been completely overhauled. Each worker has its own log queue. Workers whose logs are printed as we go are spawned in the order in which their logs must be printed, now using a TQueue instead of the old complicated LogQueueQueue mechanism. 'logThread' is replaced by 'withLogPrinter', which runs the thread that prints the log queues within a concurrent worker scope. These changes allow us to fix issues with the exception handling: we now always flush remaining log queues before exiting (even when the action throws an exception), and exceptions that arise while printing are properly rethrown. This fixes #28009. - - - - - 38a99006 by sheaf at 2026-10-26T18:19:11-04:00 Rule-based deterministic concurrent downsweep This commit rewrites downsweep as a single query-answering rule (see 'DownsweepRule') that can be executed by concurrent worker threads. The design allows every expensive operation (preprocessing files with CPP, parsing headers, reading interfaces) to be performed concurrently according to the -j<N>/-jsem flags. See Note [Rules-based downsweep] in GHC.Driver.Downsweep. The rules are run by "GHC.Driver.Concurrency.runRules" in a demand-driven way: each worker can demand other workers to run, but never wait on another worker. Every piece of work is done at most once. Fixes #27514 Memory usage can increase due to doing more work concurrently: ------------------------- Metric Increase: MultiComponentModulesRecomp MultiComponentModulesRecomp100 mhu-perf ------------------------- - - - - - 9fa7b7be by sheaf at 2026-10-26T18:19:11-04:00 Acquire/release semaphore tokens more readily This commit reworks GHC.Driver.MakeSem to use the scoped worker abstraction of GHC.Utils.Concurrent.Scope. This allows us to get rid of a lot of the tricky logic in GHC.Driver.MakeSem involving manual exception handling, thread spawning and lifetime management, etc. The main user-visible change is that token acquisition is no longer rate limited (due to a bug, there previously was a 1s rate limit on acquisition of semaphore tokens), with the release debounce period shortened from 1s to 10ms. Other changes: - With -jsem, the number of capabilities is now capped at the number of processors, bringing it in line with the -jN behaviour. - Pending jobs whose wait for a token is interrupted by an exception are now withdrawn from the jobserver, instead of acquiring a token that it hung onto without doing useful work. Fixes #27763 - - - - - c83c6733 by Alan Zimmerman at 2026-10-26T18:19:55-04:00 EPA: Update AnnClsInstDecl to contain AnnList It was already separately tracking the '{', '}' and leading ';' values. This aligns it with layout-introducing parser productions ('vocurly'), so the layout processing can be handled uniformly - - - - - ac870ad5 by Simon Jakobi at 2026-10-28T04:40:42-04:00 X86 NCG: use 32-bit POPCNT for sub-word popcounts A 32-bit write zero-extends the result to the full register, so we don't need to emit additional code for this. We also skip emitting zero-extension instructions for the argument when it is already visibly zero-extended. For the byte-wise popcount loop from #24813, this shrinks the popcount sequence from 5 instructions to 3. Assisted-by: Claude Fable 5.1 - - - - - 53087373 by Facundo Domínguez at 2026-10-28T04:41:32-04:00 Use a ShortByteString in Serialized instead of [Word8] This makes the in-memory representation 40x more compact for all values stored in annotations. A test is added to measure the representation size of annotations specified in the compiler input. The GHC API is modified. All functions that previously handled annotations in [Word8] form now use ShortByteString. The following definitions have modified types: * GHC.findGlobalAnns * GHC.Tc.Gen.Splice.getAnnotationsByTypeRep * GHC.Core.Opt.Monad.getAnnotations * GHC.Core.Opt.Monad.getFirstAnnotations * GHC.Types.Annotations.deserializeAnns * GHC.Types.Annotations.findAnns * GHC.Types.Annotations.findAnnsByTypeRep * GHC.Serialized.Serialized * GHC.Serialized.toSerialized * GHC.Serialized.fromSerialized * GHC.Serialized.serializeWithData * GHC.Serialized.deserializeWithData * GHCi.Message.ReifyAnnotations Fixes #28002. - - - - - 4cb911ea by Duncan Coutts at 2026-10-29T11:37:24+00:00 Fix a couple comments in the new selectbis I/O manager Noticed late in the review process. - - - - - af87ab12 by Duncan Coutts at 2026-10-29T11:37:24+00:00 New rts_IOManagerFeatures bitset for I/O manager features The idea is that code in the RTS and ghc-internal can use this feature bitset to do things that have to be conditional. But instead of basing it on threaded/non-threaded, win32 etc it can be based on features of the current I/O manager. In particular this will allow us to have in-RTS I/O managers when using the threaded RTS way, by changing code to use the feature test rather than testing for threaded/non-threaded. There are also numerous conditions in existing RTS and library code that could benefit from being clearly based on I/O manager features rather than on (historically) correlated aspects of the platform. For example, a major difference between WinIO and the win32-legacy I/O manager is whether it uses C lib fds or Win32 API HANDLEs. This distinction is not necessarily unique forever to these I/O managers. Future Windows I/O managers may want to take advantage of the same distinction. For example, we could have a feature to say if the I/O manager supports readiness notifications threadWaitRead# etc (win32-legacy does not support these for example), so I/O code in ghc-internal can use appropriate workarounds. Similarly, if/when we add support for async I/O, or I/O to files at given offsets (without affecting or being affected by the file pointer) then library code can consult the feature bits and either use emulated fallbacks or throw appropriate exceptions. - - - - - 0dac837a by Duncan Coutts at 2026-10-29T11:38:03+00:00 Use rts_IOManagerFeatures bitset in GHC.Internal.IO.SubSystem Establish a pattern for exposing the features. We will start using them in subsequent commits. - - - - - edef21f6 by Duncan Coutts at 2026-10-29T11:38:03+00:00 Turn rts_IOManagerIsWin32Native into a feature bit of rts_IOManagerFeatures And update where it is used in ghc-internal - - - - - 313f26bc by Duncan Coutts at 2026-10-29T11:38:03+00:00 Change G.I.Conc.IO operations to condition on iomgrInLib rather than threaded The GHC.Internal.Conc.IO defines a bunch of operations like threadDelay, threadWaitRead etc and provides conditional definitions for them in terms of the appropriate I/O manager for the platform. Some of these are static conditions (using CPP) such as Windows or JS platforms. Others are dynamic conditions, depending on whether the RTS is threaded or not. Though the threaded/non-threaded condition was really a proxy for whether the I/O manager is implemented in the library (accessed by direct calls) or within the RTS (accessed via primops). We can now change this threaded/non-threaded condition to use the new I/O manager feature bitmap. We add a 'iomgrInLib' as the inverse of 'iomgrInRTS' to make the logic clearer. Currently this will have no effect. If/when new in-RTS I/O managers are available in the threaded RTS, then the operations using this condition will Do The Right Thing™. Note that on Windows we use this to simplify the logic: it previously had to test for isWindowsNativeIO or threaded, because the WinIO I/O manager is available in the non-threaded RTS too. The WinIO I/O manager counts as an in-lib one, irrespective of threaded/non-threaded. - - - - - 0ffb2888 by Duncan Coutts at 2026-10-29T11:38:03+00:00 In G.I.Conc.IO: ensure in-lib ops are no-ops for in-RTS I/O managers The operations ensureIOManagerIsRunning, interruptIOManager and ioManagerCapabilitiesChanged are only defined for some of the in-library I/O managers. Make sure they are defined to be no-ops for the in-RTS I/O managers. - - - - - 0dcbd9b4 by Duncan Coutts at 2026-10-29T11:38:03+00:00 Replace other uses of threaded with iomgrInRTS/iomgrInLib For cases where threaded/non-threaded was being used as a proxy for whether the I/O manager is in libs or the RTS, replace it with a check on the I/O manager feature bit: iomgrInRTS/iomgrInLib. Again, more future proofing for if/when we have in-RTS I/O managers for the threaded RTS. - - - - - f5804f52 by Duncan Coutts at 2026-10-29T11:38:03+00:00 In GHC.Internal.Conc.IO more uniform dispatch to alternatives Reduce and consolotate the amount of CPP used to select between different implementations of the I/O manager actions. In particular, bring the Windows and non-Windows alternatives closer: 1. by importing the modules with the same qualified as name 2. in a couple cases by providing no-op versions of APIs so that we have a more uniform API between the two Also introduce a HAVE_IOMGR_IN_LIB define, and use that to have a static guard over the (dynamic guard) to call in-lib I/O manager functions. The pattern ends up (mostly uniformly) as: blahIOManager #if defined(HAVE_IOMGR_IN_LIB) | iomgrInLib = IOMgr.blahIOManager #endif | otherwise = ... -- the in-RTS version #endif Note: * HAVE_IOMGR_IN_LIB is a static choice * iomgrInLib is a dynamic choice Where previously there was a _lot_ more ad-hoc CPP. In particular this now covers JS and wasm more consistently, by saying they do _not_ define HAVE_IOMGR_IN_LIB, and thus only ever call the in-RTS variants. And while we're at it, make threadWaitRead, threadWaitWrite fail more politely on Windows, by throwing an exception rather than terminating the RTS! - - - - - 674 changed files: - .gitlab-ci.yml - .gitlab/hello.hs - + changelog.d/24450 - + changelog.d/25665 - + changelog.d/25850 - changelog.d/26616 - + changelog.d/26761 - + changelog.d/27368 - + changelog.d/27624 - + changelog.d/27845 - + changelog.d/27936 - + changelog.d/27971 - + changelog.d/28012 - + changelog.d/T23785 - + changelog.d/T26543 - changelog.d/T27329 - + changelog.d/T27412 - + changelog.d/T27545 - + changelog.d/T27731 - + changelog.d/T27763 - + changelog.d/T27828 - + changelog.d/T27831-bottoming-strict-field - + changelog.d/T27842-bytecode-imported-tags - + changelog.d/T27844 - + changelog.d/T27916 - + changelog.d/T28002 - + changelog.d/T28002-api - + changelog.d/T4162 - changelog.d/boxing-family - + changelog.d/clc-341-strict-mutable-data-types - + changelog.d/fix-cross-ghc-info-unit-id - changelog.d/fix-parallel-gc-selector-race-27477 - + changelog.d/hadrian-persist-flavour - + changelog.d/hpc-classinsts - + changelog.d/hpc-cleanup - + changelog.d/monad-of-no-return-canonical - + changelog.d/ncg-x86-subword-popcnt - + changelog.d/parallel-downsweep - + changelog.d/resizeMutableByteArray-pinnedness - changelog.d/tcplugin_init.md - + changelog.d/unboxed-do - changelog.d/unit-index - + changelog.d/zap-coercions - compiler/GHC.hs - compiler/GHC/Builtin.hs - compiler/GHC/Builtin/Modules.hs - compiler/GHC/Builtin/WiredIn/Ids.hs - compiler/GHC/Builtin/WiredIn/Prim.hs - compiler/GHC/Builtin/WiredIn/Types/Box.hs - compiler/GHC/Builtin/primops.txt.pp - compiler/GHC/ByteCode/Asm.hs - compiler/GHC/ByteCode/Show.hs - compiler/GHC/Cmm/CommonBlockElim.hs - compiler/GHC/Cmm/GenericOpt.hs - compiler/GHC/Cmm/UniqueRenamer.hs - compiler/GHC/CmmToAsm/CFG/Dominators.hs - compiler/GHC/CmmToAsm/Dwarf/Types.hs - compiler/GHC/CmmToAsm/Monad.hs - compiler/GHC/CmmToAsm/Reg/Graph/Spill.hs - compiler/GHC/CmmToAsm/Reg/Graph/SpillClean.hs - compiler/GHC/CmmToAsm/Reg/Graph/SpillCost.hs - compiler/GHC/CmmToAsm/Reg/Graph/Stats.hs - compiler/GHC/CmmToAsm/Reg/Linear/State.hs - compiler/GHC/CmmToAsm/Reg/Linear/Stats.hs - compiler/GHC/CmmToAsm/Reg/Liveness.hs - compiler/GHC/CmmToAsm/Wasm/Asm.hs - compiler/GHC/CmmToAsm/Wasm/Types.hs - compiler/GHC/CmmToAsm/X86/CodeGen.hs - compiler/GHC/CmmToC.hs - compiler/GHC/CmmToLlvm/CodeGen.hs - compiler/GHC/Core.hs - compiler/GHC/Core/Coercion.hs - compiler/GHC/Core/Coercion.hs-boot - compiler/GHC/Core/Coercion/Axiom.hs - compiler/GHC/Core/Coercion/Opt.hs - compiler/GHC/Core/DataCon.hs - compiler/GHC/Core/FVs.hs - compiler/GHC/Core/FamInstEnv.hs - compiler/GHC/Core/Lint.hs - + compiler/GHC/Core/Lint.hs-boot - compiler/GHC/Core/Make.hs - compiler/GHC/Core/Map/Type.hs - compiler/GHC/Core/Opt/Arity.hs - compiler/GHC/Core/Opt/CallerCC.hs - compiler/GHC/Core/Opt/CallerCC/Types.hs - compiler/GHC/Core/Opt/Exitify.hs - compiler/GHC/Core/Opt/Monad.hs - compiler/GHC/Core/Opt/Pipeline.hs - compiler/GHC/Core/Opt/Simplify/Utils.hs - compiler/GHC/Core/Opt/SpecConstr.hs - compiler/GHC/Core/Opt/Specialise.hs - compiler/GHC/Core/Reduction.hs - compiler/GHC/Core/Seq.hs - compiler/GHC/Core/SimpleOpt.hs - compiler/GHC/Core/Subst.hs - compiler/GHC/Core/TyCo/Compare.hs - compiler/GHC/Core/TyCo/FVs.hs - compiler/GHC/Core/TyCo/FVs.hs-boot - + compiler/GHC/Core/TyCo/Make.hs - + compiler/GHC/Core/TyCo/Make.hs-boot - compiler/GHC/Core/TyCo/Ppr.hs - compiler/GHC/Core/TyCo/Rep.hs - compiler/GHC/Core/TyCo/Rep.hs-boot - compiler/GHC/Core/TyCo/Subst.hs - compiler/GHC/Core/TyCo/Tidy.hs - compiler/GHC/Core/Type.hs - compiler/GHC/Core/Type.hs-boot - compiler/GHC/Core/Unify.hs - compiler/GHC/Core/Utils.hs - compiler/GHC/CoreToIface.hs - compiler/GHC/CoreToIface.hs-boot - compiler/GHC/CoreToStg.hs - + compiler/GHC/Data/Dependent.hs - compiler/GHC/Data/FastString.hs - compiler/GHC/Data/OrdList.hs - compiler/GHC/Data/OsPath.hs - compiler/GHC/Data/Pair.hs - compiler/GHC/Data/Word64Map/Internal.hs - compiler/GHC/Data/Word64Set/Internal.hs - compiler/GHC/Driver/Backpack.hs - compiler/GHC/Driver/CmdLine.hs - + compiler/GHC/Driver/Concurrency.hs - compiler/GHC/Driver/Config.hs - + compiler/GHC/Driver/Config/Concurrency.hs - compiler/GHC/Driver/Config/Core/Lint.hs - compiler/GHC/Driver/Downsweep.hs - compiler/GHC/Driver/DynFlags.hs - compiler/GHC/Driver/Env.hs - compiler/GHC/Driver/Errors/Ppr.hs - compiler/GHC/Driver/Errors/Types.hs - compiler/GHC/Driver/Flags.hs - compiler/GHC/Driver/Main/Compile.hs - compiler/GHC/Driver/Main/Passes.hs - compiler/GHC/Driver/Make.hs - compiler/GHC/Driver/MakeAction.hs - compiler/GHC/Driver/MakeFile.hs - compiler/GHC/Driver/MakeSem.hs - compiler/GHC/Driver/Pipeline.hs-boot - compiler/GHC/Driver/Pipeline/Execute.hs - compiler/GHC/Driver/Pipeline/LogQueue.hs - compiler/GHC/Driver/Pipeline/Phases.hs - compiler/GHC/Driver/Session.hs - compiler/GHC/Hs/Decls.hs - compiler/GHC/Hs/Doc.hs - compiler/GHC/Hs/Dump.hs - compiler/GHC/Hs/Expr.hs - compiler/GHC/Hs/ImpExp.hs - compiler/GHC/Hs/Lit.hs - compiler/GHC/Hs/Utils.hs - compiler/GHC/HsToCore.hs - compiler/GHC/HsToCore/Binds.hs - compiler/GHC/HsToCore/Docs.hs - compiler/GHC/HsToCore/Match/Literal.hs - compiler/GHC/HsToCore/Pmc/Types.hs - compiler/GHC/HsToCore/Quote.hs - compiler/GHC/HsToCore/Ticks.hs - compiler/GHC/Iface/Decl.hs - compiler/GHC/Iface/Errors/Types.hs - compiler/GHC/Iface/Ext/Ast.hs - compiler/GHC/Iface/Ext/Debug.hs - compiler/GHC/Iface/Ext/Types.hs - compiler/GHC/Iface/Ext/Utils.hs - compiler/GHC/Iface/Load.hs - compiler/GHC/Iface/Recomp.hs - compiler/GHC/Iface/Rename.hs - compiler/GHC/Iface/Syntax.hs - compiler/GHC/Iface/Tidy.hs - compiler/GHC/Iface/Type.hs - compiler/GHC/Iface/Type.hs-boot - compiler/GHC/IfaceToCore.hs - compiler/GHC/JS/Opt/Simple.hs - compiler/GHC/Linker/Deps.hs - compiler/GHC/Parser.y - compiler/GHC/Parser/Errors/Types.hs - compiler/GHC/Parser/Header.hs - compiler/GHC/Parser/Lexer.x - compiler/GHC/Parser/PostProcess.hs - compiler/GHC/Parser/String.hs - compiler/GHC/Rename/Env.hs - compiler/GHC/Rename/Module.hs - compiler/GHC/Rename/Names.hs - compiler/GHC/Rename/Unbound.hs - compiler/GHC/Runtime/Eval/Types.hs - compiler/GHC/Runtime/Heap/Inspect.hs - compiler/GHC/Runtime/Interpreter/JS.hs - compiler/GHC/Runtime/Loader.hs - compiler/GHC/Settings.hs - compiler/GHC/Settings/IO.hs - compiler/GHC/Stg/BcPrep.hs - compiler/GHC/Stg/Debug.hs - compiler/GHC/Stg/EnforceEpt.hs - compiler/GHC/Stg/EnforceEpt/Rewrite.hs - compiler/GHC/Stg/EnforceEpt/TagSig.hs - compiler/GHC/Stg/EnforceEpt/Types.hs - compiler/GHC/Stg/Subst.hs - compiler/GHC/StgToCmm/Prim.hs - compiler/GHC/StgToJS/Apply.hs - compiler/GHC/StgToJS/ExprCtx.hs - compiler/GHC/StgToJS/Linker/Linker.hs - compiler/GHC/StgToJS/Object.hs - compiler/GHC/StgToJS/Prim.hs - compiler/GHC/Tc/Deriv/Functor.hs - compiler/GHC/Tc/Deriv/Generate.hs - compiler/GHC/Tc/Deriv/Generics.hs - compiler/GHC/Tc/Errors.hs - compiler/GHC/Tc/Errors/Ppr.hs - compiler/GHC/Tc/Errors/Types.hs - compiler/GHC/Tc/Gen/App.hs - compiler/GHC/Tc/Gen/Export.hs - compiler/GHC/Tc/Gen/Splice.hs - compiler/GHC/Tc/Instance/Family.hs - compiler/GHC/Tc/Instance/FunDeps.hs - compiler/GHC/Tc/Module.hs - compiler/GHC/Tc/Plugin.hs - compiler/GHC/Tc/Solver/Default.hs - compiler/GHC/Tc/Solver/Equality.hs - compiler/GHC/Tc/Solver/FunDeps.hs - compiler/GHC/Tc/Solver/InertSet.hs - compiler/GHC/Tc/Solver/Monad.hs - compiler/GHC/Tc/Solver/Rewrite.hs - compiler/GHC/Tc/Solver/Solve.hs - compiler/GHC/Tc/TyCl.hs - compiler/GHC/Tc/TyCl/Instance.hs - compiler/GHC/Tc/Types.hs - compiler/GHC/Tc/Types/Constraint.hs - compiler/GHC/Tc/Types/ErrCtxt.hs - compiler/GHC/Tc/Utils/Backpack.hs - compiler/GHC/Tc/Utils/Monad.hs - compiler/GHC/Tc/Utils/TcMType.hs - compiler/GHC/Tc/Utils/TcType.hs - compiler/GHC/Tc/Utils/Unify.hs - compiler/GHC/Tc/Zonk/Env.hs - compiler/GHC/Tc/Zonk/Monad.hs - compiler/GHC/Tc/Zonk/TcType.hs - compiler/GHC/Tc/Zonk/Type.hs - compiler/GHC/ThToHs.hs - compiler/GHC/Types/Annotations.hs - compiler/GHC/Types/Arity.hs - compiler/GHC/Types/Basic.hs - compiler/GHC/Types/Cpr.hs - compiler/GHC/Types/Demand.hs - compiler/GHC/Types/Error/Codes.hs - compiler/GHC/Types/Hint.hs - compiler/GHC/Types/Hint/Ppr.hs - compiler/GHC/Types/Id/Info.hs - compiler/GHC/Types/Name/Env.hs - compiler/GHC/Types/Name/Occurrence.hs - compiler/GHC/Types/Unique.hs - compiler/GHC/Types/Unique/DSM.hs - compiler/GHC/Types/Unique/DSet.hs - compiler/GHC/Types/Unique/FM.hs - compiler/GHC/Types/Unique/Map.hs - compiler/GHC/Types/Unique/Set.hs - compiler/GHC/Types/Unique/Supply.hs - compiler/GHC/Types/UnresolvedImport.hs - compiler/GHC/Types/Var.hs - compiler/GHC/Types/Var.hs-boot - compiler/GHC/Types/Var/FV.hs - compiler/GHC/Types/Var/Set.hs - compiler/GHC/Unit/External/Database.hs - compiler/GHC/Unit/External/ModuleOrigin.hs - compiler/GHC/Unit/External/Visibility.hs - compiler/GHC/Unit/Finder.hs - + compiler/GHC/Unit/Finder/Cache.hs - compiler/GHC/Unit/Finder/Types.hs - compiler/GHC/Unit/Home.hs - compiler/GHC/Unit/Module.hs - compiler/GHC/Unit/Module/Env.hs - compiler/GHC/Unit/Module/Graph.hs - compiler/GHC/Unit/Module/ModSummary.hs - + compiler/GHC/Unit/Module/Name.hs - compiler/GHC/Unit/Module/WholeCoreBindings.hs - compiler/GHC/Unit/Parser.hs - compiler/GHC/Unit/State.hs - compiler/GHC/Unit/Types.hs - compiler/GHC/Unit/Types.hs-boot - compiler/GHC/Utils/Binary.hs - + compiler/GHC/Utils/Concurrent/Scope.hs - compiler/GHC/Utils/Monad/Codensity.hs - + compiler/GHC/Utils/Monad/StrictIdentity.hs - + compiler/GHC/Utils/Monad/StrictReader.hs - compiler/GHC/Utils/Monad/State/Strict.hs → compiler/GHC/Utils/Monad/StrictState.hs - compiler/GHC/Utils/Outputable.hs - compiler/GHC/Utils/Panic.hs - compiler/GHC/Utils/Ppr/Colour.hs - compiler/GHC/Utils/TmpFs.hs - compiler/Language/Haskell/Syntax.hs - compiler/Language/Haskell/Syntax/Decls.hs - compiler/Language/Haskell/Syntax/Expr.hs - compiler/Language/Haskell/Syntax/Extension.hs - compiler/Language/Haskell/Syntax/ImpExp.hs - compiler/Language/Haskell/Syntax/Lit.hs - compiler/Language/Haskell/Syntax/Module/Name.hs - compiler/Language/Haskell/Syntax/Specificity.hs - compiler/Language/Haskell/Syntax/Text.hs - compiler/ghc.cabal.in - distrib/configure.ac.in - docs/users_guide/extending_ghc.rst - docs/users_guide/exts/literals.rst - docs/users_guide/exts/qualified_do.rst - docs/users_guide/exts/type_families.rst - docs/users_guide/using-warnings.rst - ghc/GHCi/UI.hs - ghc/GHCi/UI/Info.hs - ghc/Main.hs - ghc/ghc-bin.cabal.in - hadrian/bindist/Makefile - hadrian/bindist/config.mk.in - hadrian/doc/flavours.md - hadrian/src/Hadrian/Haskell/Cabal/Parse.hs - hadrian/src/Hadrian/Haskell/Cabal/Type.hs - hadrian/src/Main.hs - hadrian/src/Rules.hs - hadrian/src/Rules/BinaryDist.hs - hadrian/src/Rules/Documentation.hs - hadrian/src/Rules/Generate.hs - hadrian/src/Rules/Selftest.hs - hadrian/src/Settings.hs - hadrian/src/Settings/Builders/Ar.hs - hadrian/src/Settings/Warnings.hs - libraries/base/base.cabal.in - libraries/base/changelog.md - libraries/base/src/Control/Concurrent/Chan.hs - + libraries/base/src/Control/Concurrent/Chan/Strict.hs - libraries/base/src/Control/Concurrent/MVar.hs - + libraries/base/src/Control/Concurrent/MVar/Strict.hs - libraries/base/src/Data/IORef.hs - + libraries/base/src/Data/IORef/Strict.hs - libraries/base/src/GHC/Base.hs - libraries/base/src/GHC/Exts.hs - libraries/base/src/System/Timeout.hs - + libraries/base/tests/IO/T27971.hs - + libraries/base/tests/IO/T27971.stdout - libraries/base/tests/IO/all.T - libraries/base/tests/Memo1.lhs - libraries/base/tests/Memo2.lhs - libraries/base/tests/all.T - libraries/containers - libraries/ghc-boot/GHC/Serialized.hs - libraries/ghc-internal/CHANGELOG.md - libraries/ghc-internal/ghc-internal.cabal.in - libraries/ghc-internal/src/GHC/Internal/Base.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Backend/GMP.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Backend/Native.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/BigNat.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Integer.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Natural.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Primitives.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/WordArray.hs - libraries/ghc-internal/src/GHC/Internal/Conc/IO.hs - libraries/ghc-internal/src/GHC/Internal/Conc/POSIX.hs - libraries/ghc-internal/src/GHC/Internal/Conc/Windows.hs - libraries/ghc-internal/src/GHC/Internal/Event/Thread.hs - libraries/ghc-internal/src/GHC/Internal/Event/Windows.hsc - libraries/ghc-internal/src/GHC/Internal/Event/Windows/FFI.hsc - libraries/ghc-internal/src/GHC/Internal/IO/SubSystem.hs - libraries/ghc-internal/src/GHC/Internal/IO/Windows/Handle.hsc - + libraries/ghc-internal/src/GHC/Internal/Unboxed/Do.hs - libraries/ghci/GHCi/Message.hs - libraries/ghci/GHCi/TH.hs - libraries/ghci/GHCi/TH/Binary.hs - linters/lint-codes/LintCodes/Static.hs - m4/subst_tooldir.m4 - rts/IOManager.c - rts/IOManager.h - rts/PrimOps.cmm - rts/RtsStartup.c - rts/RtsSymbols.c - rts/include/stg/MiscClosures.h - rts/js/environment.js - rts/posix/SelectBis.c - testsuite/driver/runtests.py - testsuite/tests/ado/ado-optimal.hs - testsuite/tests/ado/ado001.hs - testsuite/tests/annotations/should_run/Makefile - + testsuite/tests/annotations/should_run/T16841.hs - + testsuite/tests/annotations/should_run/T16841.stdout - testsuite/tests/annotations/should_run/all.T - + testsuite/tests/backpack/should_compile/T27835.bkp - + testsuite/tests/backpack/should_compile/T27835.stderr - testsuite/tests/backpack/should_compile/all.T - + testsuite/tests/bytecode/T27842.hs - + testsuite/tests/bytecode/T27842.stdout - + testsuite/tests/bytecode/T27842A.hs - + testsuite/tests/bytecode/T27842N.hs - testsuite/tests/bytecode/all.T - + testsuite/tests/codeGen/should_compile/T27368.hs - testsuite/tests/codeGen/should_compile/all.T - + testsuite/tests/codeGen/should_gen_asm/T24813.asm - + testsuite/tests/codeGen/should_gen_asm/T24813.cmm - testsuite/tests/codeGen/should_gen_asm/all.T - + testsuite/tests/codeGen/should_run/T27624.hs - + testsuite/tests/codeGen/should_run/T27624.stdout - testsuite/tests/codeGen/should_run/all.T - testsuite/tests/concurrent/T13615/Parallel.hs - testsuite/tests/concurrent/should_run/T21651.hs - testsuite/tests/concurrent/should_run/setnumcapabilities001.hs - testsuite/tests/corelint/T27374.hs - testsuite/tests/count-deps/CountDepsAst.stdout - testsuite/tests/count-deps/CountDepsParser.stdout - testsuite/tests/cpranal/should_run/T20598a.hs - testsuite/tests/default/default-fail05.stderr - testsuite/tests/dependent/should_compile/T14729.stderr - testsuite/tests/deriving/should_compile/T13758.hs - testsuite/tests/deriving/should_run/all.T - + testsuite/tests/deriving/should_run/drvrun-canonical-methods.hs - + testsuite/tests/deriving/should_run/drvrun-canonical-methods.stderr - + testsuite/tests/deriving/should_run/drvrun-canonical-methods.stdout - testsuite/tests/determinism/determ017/A.hs - testsuite/tests/diagnostic-codes/codes.stdout - testsuite/tests/dmdanal/should_compile/T9208.hs - + testsuite/tests/dmdanal/should_run/T27845.hs - + testsuite/tests/dmdanal/should_run/T27845.stdout - + testsuite/tests/dmdanal/should_run/T27845_Lib.hs - + testsuite/tests/dmdanal/should_run/T27845b.hs - + testsuite/tests/dmdanal/should_run/T27845b.stdout - + testsuite/tests/dmdanal/should_run/T27845b_Lib.hs - testsuite/tests/dmdanal/should_run/all.T - testsuite/tests/driver/Makefile - testsuite/tests/driver/T27461/Makefile - + testsuite/tests/driver/T27461/T27461c.stderr - testsuite/tests/driver/T27461/all.T - + testsuite/tests/driver/T27461/src/Wrong.hs - + testsuite/tests/driver/T27836/A.hs - + testsuite/tests/driver/T27836/B.hs - + testsuite/tests/driver/T27836/C.hs - + testsuite/tests/driver/T27836/Makefile - + testsuite/tests/driver/T27836/T27836.stdout - + testsuite/tests/driver/T27836/all.T - + testsuite/tests/driver/T27836/unitA - + testsuite/tests/driver/T27836/unitB - + testsuite/tests/driver/T27836/unitC - + testsuite/tests/driver/T28009.hs - testsuite/tests/driver/all.T - testsuite/tests/generics/T27245.hs - testsuite/tests/generics/T27245.stdout - + testsuite/tests/ghc-api/T4162.hs - + testsuite/tests/ghc-api/T4162.stdout - testsuite/tests/ghc-api/all.T - testsuite/tests/ghc-api/downsweep/IncrementalDownsweep.hs - testsuite/tests/ghc-api/downsweep/OldModLocation.hs - testsuite/tests/ghc-api/downsweep/PartialDownsweep.hs - testsuite/tests/ghc-api/exactprint/T22919.stderr - testsuite/tests/ghc-api/exactprint/Test20239.stderr - testsuite/tests/ghc-api/exactprint/ZeroWidthSemi.stderr - testsuite/tests/ghc-api/fixed-nodes/FixedNodes.hs - testsuite/tests/ghc-api/fixed-nodes/InterfaceModuleGraph.hs - testsuite/tests/ghc-api/fixed-nodes/ModuleGraphInvariants.hs - testsuite/tests/ghci.debugger/scripts/T14628.stderr - testsuite/tests/ghci/prog-mhu001/prog-mhu001c.stdout - testsuite/tests/ghci/prog-mhu002/prog-mhu002d.stdout - testsuite/tests/ghci/prog020/ghci.prog020a.stderr - testsuite/tests/ghci/prog020/ghci.prog020b.stderr - testsuite/tests/ghci/prog021/prog021a.stderr - testsuite/tests/ghci/prog021/prog021b.stderr - testsuite/tests/ghci/prog022/ghci.prog022a.stderr - testsuite/tests/ghci/prog022/ghci.prog022b.stderr - testsuite/tests/ghci/prog022/ghci.prog022c.stderr - testsuite/tests/ghci/prog022/ghci.prog022d.stderr - testsuite/tests/ghci/prog022/ghci.prog022e.stderr - testsuite/tests/ghci/prog022/ghci.prog022f.stderr - testsuite/tests/ghci/scripts/Defer02.stderr - testsuite/tests/ghci/scripts/T16575.script - testsuite/tests/ghci/scripts/T16575.stdout - testsuite/tests/ghci/scripts/T16804.stdout - testsuite/tests/ghci/scripts/T20150.stdout - + testsuite/tests/ghci/scripts/T27412.hs - + testsuite/tests/ghci/scripts/T27412.script - + testsuite/tests/ghci/scripts/T27412.stdout - testsuite/tests/ghci/scripts/all.T - testsuite/tests/haddock/perf/Fold.hs - testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T17544_kw.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T24221.stderr - testsuite/tests/hiefile/should_compile/hie002.hs - testsuite/tests/hiefile/should_run/T24544.hs - + testsuite/tests/hpc/instmeths/instmeths.hs - + testsuite/tests/hpc/instmeths/instmeths.stdout - + testsuite/tests/hpc/instmeths/test.T - testsuite/tests/hpc/raytrace/Eval.hs - + testsuite/tests/impredicative/T27917.hs - testsuite/tests/impredicative/all.T - + testsuite/tests/indexed-types/should_compile/T27828.hs - + testsuite/tests/indexed-types/should_compile/T27828a.hs - + testsuite/tests/indexed-types/should_compile/T27828b.hs - testsuite/tests/indexed-types/should_compile/T3787.hs - + testsuite/tests/indexed-types/should_compile/ZapDeps.hs - testsuite/tests/indexed-types/should_compile/all.T - testsuite/tests/indexed-types/should_fail/T1897b.stderr - testsuite/tests/indexed-types/should_fail/T7729.hs - testsuite/tests/indexed-types/should_fail/T7729a.hs - testsuite/tests/indexed-types/should_fail/T9662.stderr - testsuite/tests/interface-stability/base-exports.stdout - testsuite/tests/interface-stability/base-exports.stdout-javascript-unknown-ghcjs - testsuite/tests/interface-stability/base-exports.stdout-mingw32 - testsuite/tests/interface-stability/ghc-experimental-exports.stdout - testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32 - testsuite/tests/interface-stability/ghc-prim-exports.stdout - testsuite/tests/interface-stability/ghc-prim-exports.stdout-mingw32 - testsuite/tests/javascript/Makefile - + testsuite/tests/javascript/T25665.hs - + testsuite/tests/javascript/T25665.pkg - + testsuite/tests/javascript/T25665.stdout - + testsuite/tests/javascript/T25665Lib.hs - + testsuite/tests/javascript/T25665_c.c - + testsuite/tests/javascript/T25665_js.js - testsuite/tests/javascript/all.T - testsuite/tests/module/mod130.stderr - testsuite/tests/module/mod133.hs - testsuite/tests/module/mod185.stderr - + testsuite/tests/numeric/should_run/T27545.hs - + testsuite/tests/numeric/should_run/T27545.stdout - testsuite/tests/numeric/should_run/all.T - testsuite/tests/overloadedstrings/should_fail/T25926.stderr - testsuite/tests/overloadedstrings/should_fail/T27124.stderr - testsuite/tests/package/T4806a.stderr - testsuite/tests/package/all.T - testsuite/tests/parser/should_compile/DumpParsedAst.stderr - testsuite/tests/parser/should_compile/DumpParsedAstComments.stderr - testsuite/tests/parser/should_compile/DumpRenamedAst.stderr - testsuite/tests/parser/should_compile/DumpSemis.stderr - testsuite/tests/parser/should_compile/KindSigs.stderr - testsuite/tests/parser/should_compile/T14189.stderr - testsuite/tests/parser/should_compile/T15323.stderr - testsuite/tests/parser/should_compile/T20452.stderr - testsuite/tests/parser/should_compile/T20718.stderr - testsuite/tests/parser/should_compile/T20718b.stderr - testsuite/tests/parser/should_compile/T20846.stderr - testsuite/tests/parser/should_compile/T23315/T23315.stderr - testsuite/tests/patsyn/should_fail/records-nofieldselectors.stderr - testsuite/tests/perf/compiler/T13253.hs - testsuite/tests/perf/compiler/T14683.hs - testsuite/tests/perf/compiler/T16473.hs - testsuite/tests/perf/compiler/T18730_A.hs - testsuite/tests/perf/compiler/T25723.hs - + testsuite/tests/plugins/T26761.hs - + testsuite/tests/plugins/T26761.stderr - + testsuite/tests/plugins/T26761Plugin.hs - + testsuite/tests/plugins/T26761a.hs - testsuite/tests/plugins/all.T - testsuite/tests/plugins/defaulting-plugin/DefaultLifted.hs - testsuite/tests/pmcheck/should_compile/T11276.hs - + testsuite/tests/primops/should_run/ResizeByteArrayPinnedness.hs - + testsuite/tests/primops/should_run/ResizeByteArrayPinnedness.stdout - testsuite/tests/primops/should_run/all.T - testsuite/tests/printer/AnnotationNoListTuplePuns.stdout - testsuite/tests/printer/T17697.stderr - testsuite/tests/printer/T18791.stderr - testsuite/tests/printer/Test20297.stdout - testsuite/tests/printer/Test24533.stdout - testsuite/tests/profiling/should_compile/T19894/StreamK.hs - testsuite/tests/profiling/should_run/T3001-2.hs - testsuite/tests/profiling/should_run/caller-cc/CallerCcParser.hs - testsuite/tests/programs/galois_raytrace/Eval.hs - testsuite/tests/qualifieddo/should_compile/all.T - + testsuite/tests/qualifieddo/should_compile/qdocompile003.hs - + testsuite/tests/qualifieddo/should_compile/qdocompile003.stderr - testsuite/tests/qualifieddo/should_fail/all.T - + testsuite/tests/qualifieddo/should_fail/qdofail006.hs - + testsuite/tests/qualifieddo/should_fail/qdofail006.stderr - testsuite/tests/qualifieddo/should_run/all.T - + testsuite/tests/qualifieddo/should_run/qdorun008.hs - + testsuite/tests/qualifieddo/should_run/qdorun008.stdout - testsuite/tests/rebindable/rebindable2.hs - + testsuite/tests/rename/should_fail/Noncanonical.hs - + testsuite/tests/rename/should_fail/Noncanonical.stderr - testsuite/tests/rename/should_fail/T15607.stderr - testsuite/tests/rename/should_fail/T19843b.stderr - testsuite/tests/rename/should_fail/all.T - testsuite/tests/rename/should_fail/rnfail026.hs - testsuite/tests/rep-poly/RepPolyRule3.stderr - + testsuite/tests/rts/T24450.hs - + testsuite/tests/rts/T24450.stderr - + testsuite/tests/rts/T27526.hs - + testsuite/tests/rts/T27526.stdout - + testsuite/tests/rts/T28012.hs - + testsuite/tests/rts/T28012.stdout - testsuite/tests/rts/all.T - testsuite/tests/rts/ipe/IpeStats/Fold.hs - testsuite/tests/safeHaskell/ghci/p4.stderr - testsuite/tests/safeHaskell/ghci/p6.stderr - testsuite/tests/simplCore/should_compile/T14908_Deps.hs - testsuite/tests/simplCore/should_compile/T22375.stderr - testsuite/tests/simplCore/should_compile/T22375DataFamily.stderr - testsuite/tests/simplCore/should_compile/T22611.hs - testsuite/tests/simplCore/should_compile/T22611.stderr - + testsuite/tests/simplCore/should_compile/T22611a.hs - testsuite/tests/simplCore/should_compile/T26051.stderr - + testsuite/tests/simplCore/should_compile/T27910.hs - + testsuite/tests/simplCore/should_compile/T27910.stderr - + testsuite/tests/simplCore/should_compile/T27936.hs - testsuite/tests/simplCore/should_compile/all.T - testsuite/tests/simplCore/should_compile/simpl017.hs - testsuite/tests/simplStg/should_compile/all.T - + testsuite/tests/simplStg/should_compile/inferTags005.hs - + testsuite/tests/simplStg/should_compile/inferTags005.stderr - + testsuite/tests/simplStg/should_compile/inferTags006.hs - + testsuite/tests/simplStg/should_compile/inferTags006.stderr - + testsuite/tests/simplStg/should_run/InferTags007_A.hs - + testsuite/tests/simplStg/should_run/T27831.hs - + testsuite/tests/simplStg/should_run/T27831.stdout - + testsuite/tests/simplStg/should_run/T27857.hs - + testsuite/tests/simplStg/should_run/T27857.stdout - testsuite/tests/simplStg/should_run/all.T - + testsuite/tests/simplStg/should_run/inferTags007.hs - + testsuite/tests/simplStg/should_run/inferTags007.stdout - testsuite/tests/splice-imports/SI35.hs - testsuite/tests/typecheck/should_compile/DeepSubsumption02.hs - testsuite/tests/typecheck/should_compile/LocalGivenEqs.hs - testsuite/tests/typecheck/should_compile/T11254.stderr - testsuite/tests/typecheck/should_compile/T15370.stderr - + testsuite/tests/typecheck/should_compile/T26543_orig.hs - + testsuite/tests/typecheck/should_compile/T27916.hs - testsuite/tests/typecheck/should_compile/T4524.hs - testsuite/tests/typecheck/should_compile/T4969.hs - testsuite/tests/typecheck/should_compile/all.T - testsuite/tests/typecheck/should_compile/valid_hole_fits.stderr - testsuite/tests/typecheck/should_fail/ClassOperator.stderr - testsuite/tests/typecheck/should_fail/DoExpansion2.stderr - testsuite/tests/typecheck/should_fail/DoExpansion3.stderr - testsuite/tests/typecheck/should_fail/FunDepOrigin1b.hs - testsuite/tests/typecheck/should_fail/T12785b.stderr - testsuite/tests/typecheck/should_fail/T18851.stderr - + testsuite/tests/typecheck/should_fail/T18851e.hs - + testsuite/tests/typecheck/should_fail/T18851e.stderr - + testsuite/tests/typecheck/should_fail/T18851f.hs - + testsuite/tests/typecheck/should_fail/T18851f.stderr - testsuite/tests/typecheck/should_fail/T22707.hs - testsuite/tests/typecheck/should_fail/T22940.stderr - testsuite/tests/typecheck/should_fail/T25950.stderr - + testsuite/tests/typecheck/should_fail/T26543a.hs - + testsuite/tests/typecheck/should_fail/T26543a.stderr - testsuite/tests/typecheck/should_fail/T26823.stderr - + testsuite/tests/typecheck/should_fail/T27731.hs - + testsuite/tests/typecheck/should_fail/T27731.stderr - + testsuite/tests/typecheck/should_fail/T27844.hs - + testsuite/tests/typecheck/should_fail/T27844.stderr - testsuite/tests/typecheck/should_fail/T5691.stderr - testsuite/tests/typecheck/should_fail/T7019.hs - testsuite/tests/typecheck/should_fail/T7437.stderr - testsuite/tests/typecheck/should_fail/T8450.stderr - testsuite/tests/typecheck/should_fail/T8603.hs - testsuite/tests/typecheck/should_fail/all.T - testsuite/tests/typecheck/should_fail/mc24.stderr - testsuite/tests/typecheck/should_fail/tcfail019.stderr - testsuite/tests/typecheck/should_fail/tcfail189.stderr - + testsuite/tests/warnings/should_compile/RedundantCanonicalMethods.hs - + testsuite/tests/warnings/should_compile/RedundantCanonicalMethods.stderr - testsuite/tests/warnings/should_compile/WarnNoncanonical.hs - testsuite/tests/warnings/should_compile/WarnNoncanonical.stderr - testsuite/tests/warnings/should_compile/all.T - utils/check-exact/ExactPrint.hs - utils/check-exact/Parsers.hs - utils/check-exact/Utils.hs - utils/check-ppr/Main.hs - utils/count-deps/Main.hs - utils/dump-decls/Main.hs - utils/haddock/haddock-api/haddock-api.cabal - utils/haddock/haddock-api/src/Haddock.hs - utils/haddock/haddock-api/src/Haddock/Backends/Hoogle.hs - utils/haddock/haddock-api/src/Haddock/Backends/Hyperlinker/Renderer.hs - utils/haddock/haddock-api/src/Haddock/Backends/Hyperlinker/Utils.hs - utils/haddock/haddock-api/src/Haddock/Backends/LaTeX.hs - utils/haddock/haddock-api/src/Haddock/Backends/Xhtml/Decl.hs - utils/haddock/haddock-api/src/Haddock/Convert.hs - utils/haddock/haddock-api/src/Haddock/Interface.hs - utils/haddock/haddock-api/src/Haddock/Interface/ParseModuleHeader.hs - utils/haddock/haddock-api/src/Haddock/Interface/Rename.hs - utils/haddock/haddock-api/src/Haddock/Types.hs - utils/haddock/haddock-api/src/Haddock/Utils/Json.hs - utils/haddock/haddock-api/test/Haddock/Backends/Hyperlinker/ParserSpec.hs - utils/haddock/haddock-library/haddock-library.cabal - utils/haddock/haddock.cabal - utils/runghc/Main.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/ffcb0c306b09664143f54387253f14… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/ffcb0c306b09664143f54387253f14… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0

[Git][ghc/ghc][wip/marge_bot_batch_merge_job] 3 commits: testsuite: Fix python type errors
by Marge Bot (＠marge-bot) 29 Oct '26

29 Oct '26

Marge Bot pushed to branch wip/marge_bot_batch_merge_job at Glasgow Haskell Compiler / GHC Commits: da4ed989 by David Eichmann at 2026-10-29T07:28:09-04:00 testsuite: Fix python type errors These errors appear when using the VSCode python extension which uses Pylance under the hood. - - - - - 23203402 by David Eichmann at 2026-10-29T07:28:09-04:00 Add test modifier to test over all IO Managers - - - - - 6e8ab2f3 by Alan Zimmerman at 2026-10-29T07:28:09-04:00 EPA: Tidy up ExactPrint.markAnnListA' It takes an action which can modify the AnnList, but this is never used. So remove it. - - - - - 8 changed files: - rts/RtsUtils.c - testsuite/config/ghc - testsuite/driver/runtests.py - testsuite/driver/testglobals.py - testsuite/driver/testlib.py - testsuite/driver/testutil.py - testsuite/tests/rts/all.T - utils/check-exact/ExactPrint.hs Changes: ===================================== rts/RtsUtils.c ===================================== @@ -397,6 +397,7 @@ void printRtsInfo(const RtsConfig rts_config) { rts_config.rts_opts != NULL ? rts_config.rts_opts : ""); selectIOManager(); /* resolve the io-manager, accounting for flags */ mkRtsInfoPair("I/O manager default", showIOManager()); + mkRtsInfoPair("I/O managers", &IOMGRS_ENABLED_STR[1]); // skips the initial ' ' character in IOMGRS_ENABLED_STR printf(" ]\n"); } ===================================== testsuite/config/ghc ===================================== @@ -257,7 +257,9 @@ def get_compiler_info(): ) except Exception as err: - print("Exception thrown in testsuite/config/ghc.get_compiler_info: %s", err) + # We don't throw so that we can still discover tests before we've built + # GHC (i.e. config.compiler doesn't yet exist). + print("Exception thrown in testsuite/config/ghc.get_compiler_info: ", err) else: res = p.returncode == 0 @@ -315,6 +317,58 @@ def get_compiler_info(): config.ghc_th_way = "normal" config.ghc_th_hi_ext = "hi" + # I/O manager discovery. + # + # Which I/O managers (GHC's `--io-manager` RTS flag) a given RTS build + # supports depends on the target platform and on whether the RTS is the + # threaded or non-threaded flavour (see rts/IOManager.h's + # IOMGR_ENABLED_* macros). Rather than duplicate that platform/way + # dependent logic here, we ask the RTS itself using `+RTS --info -RTS` + # and extract the "I/O managers" value from stdout. + def discover_io_managers_generic(extra_flags): + import tempfile + + test_src = 'main :: IO ()\nmain = return ()\n' + + res = [None] + + with tempfile.TemporaryDirectory() as d: + src = Path(d) / 'IOMgrProbe.hs' + src.write_text(test_src) + exe = Path(d) / ('IoMgrProbe' + config.exeext) + + # -rtsopts is needed so that the resulting binary will even + # consider an --io-manager= flag (rather than immediately + # rejecting all "unsafe" RTS options). + cmd = '{} -v0 -rtsopts -o {} {} {}'.format( + config.compiler, + exe, + src, + ' '.join(extra_flags)) + try: + cp = subprocess.run( + cmd, + shell=True, + cwd=d, + stdout=subprocess.DEVNULL, + stderr=None if config.verbose >= 3 else subprocess.DEVNULL) + if cp.returncode != 0 or not exe.exists(): + raise Exception("Error in testsuite/config/ghc.discover_io_managers: failed to compile with: {}".format(cmd)) + res = list(discover_io_managers(exe, d)) + + except Exception as err: + # We don't throw so that we can still discover tests before we've built + # GHC (i.e. config.compiler doesn't yet exist). + print("Exception thrown in testsuite/config/ghc.discover_io_managers_generic: ", err) + + return res + + config.io_managers_nonthreaded = discover_io_managers_generic([]) + config.io_managers_threaded = ( + discover_io_managers_generic(['-threaded']) if config.ghc_with_threaded_rts + else []) + config.io_managers_all = list(set(config.io_managers_nonthreaded).union(set(config.io_managers_threaded))) + config.cmm_cpp_is_gcc = test_compile( ['-c'], test_filename='test.cmm', ===================================== testsuite/driver/runtests.py ===================================== @@ -28,7 +28,7 @@ import subprocess import asyncio -from testutil import getStdout, str_warn, str_info, print_table, shorten_metric_name, str_removeprefix +from testutil import discover_io_managers, getStdout, str_warn, str_info, print_table, shorten_metric_name, str_removeprefix from testglobals import getConfig, ghc_env, TestConfig, t, \ TestOptions, brokens, PerfMetric from my_typing import TestName ===================================== testsuite/driver/testglobals.py ===================================== @@ -126,6 +126,15 @@ class TestConfig: self.way_flags = {} # type: Dict[WayName, List[str]] self.way_rts_flags = {} # type: Dict[WayName, List[str]] + # I/O managers (GHC's `--io-manager` RTS flag) supported by the threaded runtime. + self.io_managers_threaded = [] # type: List[str] + + # I/O managers (GHC's `--io-manager` RTS flag) supported by the non-threaded runtime. + self.io_managers_nonthreaded = [] # type: List[str] + + # All I/O managers (GHC's `--io-manager` RTS flag) + self.io_managers_all = [] # type: List[str] + # Do we have a functional LLVM toolchain? self.have_llvm = False @@ -227,7 +236,7 @@ class TestConfig: self.allowed_perf_changes = {} # The test environment. - self.test_env = 'local' + self.test_env = TestEnv('local') # terminal supports colors self.supports_colors = False @@ -367,6 +376,14 @@ class TestOptions: # add these ways to the default set self.extra_ways = [] # type: List[WayName] + # IO Managers to test (passed as the --io-manager RTS option) + # * [None] - only default IO Manager + # * [...] - list of io managers e.g. "mio" or "poll" + self.io_managers = [None] # type: List[Optional[str]] + + # IO Manager used in the current test case. Set before each test case. None means default IO Manager + self.io_manager_current = None # type: Optional[str] + # the result we normally expect for this test self.expect = 'pass' ===================================== testsuite/driver/testlib.py ===================================== @@ -23,7 +23,7 @@ import subprocess from testglobals import config, ghc_env, default_testopts, brokens, t, \ TestRun, TestResult, TestOptions, PerfMetric -from testutil import strip_quotes, lndir, link_or_copy_file, passed, \ +from testutil import discover_io_managers, strip_quotes, lndir, link_or_copy_file, passed, \ failBecause, residency_testing_metrics, \ stable_perf_counters, \ PassFail, badResult, str_warn, str_removeprefix @@ -190,7 +190,7 @@ def no_lint( name, opts ): def stage1(name, opts): # See Note [Why is there no stage1 setup function?] - framework_fail(name, 'stage1 setup function does not exist', + framework_fail(name, None, 'add your test to testsuite/tests/stage1 instead') # Note [Why is there no stage1 setup function?] @@ -341,7 +341,7 @@ def req_th( name, opts ): req_interp(name, opts); if ghc_dynamic(): - return _omit_ways(name, opts, ['profasm', 'profthreaded']) + return _omit_ways(name, opts, [WayName('profasm'), WayName('profthreaded')]) def req_plugins( name, opts ): """ @@ -355,7 +355,7 @@ def req_plugins( name, opts ): req_interp(name, opts) # Plugins aren't supported with the external interpreter (#14335) - expect_broken_for(14335,['ext-interp'])(name,opts) + expect_broken_for(IssueNumber(14335),[WayName('ext-interp')])(name,opts) if config.cross: opts.skip = True @@ -569,6 +569,34 @@ def extra_ways( ways: List[WayName] ): return helper +# ----- + +def _io_managers( name: TestName, opts, io_mgrs: List[str] ) -> None: + opts.io_managers = io_mgrs + +def io_managers( io_mgrs: List[str] ): + """ + Further split test cases into the given IO Managers if available. + Available IO Managers are determined by the test case's way (threaded/non-threads) and OS. + You'll likely want to use `extra_ways(['threaded2'])` if you want to test threaded IO Managers (e.g. mio). + """ + return lambda name, opts: _io_managers(name, opts, io_mgrs) + +def all_available_io_managers( name: TestName, opts ): + """ + Further split test cases into all available IO Managers. + Available IO Managers are determined by the test case's way (threaded/non-threads) and OS. + You'll likely want to use `extra_ways(['threaded2'])` to ensure threaded IO Managers are tested (e.g. mio). + """ + _io_managers(name, opts, config.io_managers_all) + +def all_available_io_managers_except(exclude: List[str]): + """ + Further split test cases into all available (except the given excluded) IO Managers. + Available IO Managers are determined by the test case's way (threaded/non-threads) and OS. + You'll likely want to use `extra_ways(['threaded2'])` to ensure threaded IO Managers are tested (e.g. mio). + """ + return lambda name, opts: _io_managers(name, opts, list(set(config.io_managers_all) - set(exclude))) # ----- @@ -661,18 +689,15 @@ def collect_size ( deviation, path ): def collect_size_func ( deviation, path_func ): return collect_generic_stat ( 'size', deviation, lambda way: os.path.getsize(in_testdir(path_func())) ) -def get_dir_size(path): +def get_dir_size(path) -> int: total = 0 - try: - with os.scandir(path) as it: - for entry in it: - if entry.is_file(): - total += entry.stat().st_size - elif entry.is_dir(): - total += get_dir_size(entry.path) - return total - except FileNotFoundError: - print("Exception: Could not find: " + path) + with os.scandir(path) as it: + for entry in it: + if entry.is_file(): + total += entry.stat().st_size + elif entry.is_dir(): + total += get_dir_size(entry.path) + return total def collect_size_dir ( deviation, path ): return collect_size_dir_func ( deviation, lambda: path ) @@ -790,7 +815,7 @@ def find_non_inplace_so(lib): return _find_so(lib,path_from_ghcPkg(lib, "dynamic-library-dirs"),False) -def collect_generic_stat ( metric, deviation: Optional[int], get_stat: Callable[[WayName], str]): +def collect_generic_stat ( metric, deviation: Optional[int], get_stat: Callable[[WayName], Any]): """ Define a generic stat test, which computes the statistic by calling the function given as the third argument. @@ -1135,7 +1160,7 @@ def _collect_compiler_perf_counters(counters: Set[str], deviation: Optional[int] # Slightly hacky, we need the requested perf_counters in 'simple_run'. # Thus, we have to globally register these counters opts.compiler_perf_counters += list(counters) - _collect_stats(name, opts, counters, deviation, False, True, True) + _collect_stats(name, opts, counters, deviation, None, True, True) return f @@ -1334,21 +1359,22 @@ def check_errmsg(needle): # grep_errmsg(regex,[groups]) # If groups are given, return only the matched groups # that matches the regex. -def grep_errmsg_norm(needle:str, groups = None): - - def get_match(str:str): - m = re.search(needle,str) - if m: - return "".join([m.group(g) for g in groups if m.group(g) is not None]) - else: - return None +def grep_errmsg_norm(needle:str, groups_may = None): - def norm(str) -> str: - if groups == None: + def norm(string) -> str: + if groups_may == None: return "".join( filter(lambda l: re.search(needle,l), - str.splitlines(True))) + string.splitlines(True))) else: - matches = [get_match(x) for x in str.splitlines(True)] + groups = groups_may + def get_match(string: str): + m = re.search(needle, string) + if m: + return "".join([m.group(g) for g in groups if m.group(g) is not None]) + else: + return None + + matches = [get_match(x) for x in string.splitlines(True)] res = "\n".join([x for x in matches if x]) return res return norm @@ -1573,6 +1599,15 @@ do_not_copy = ('.hi', '.o', '.dyn_hi' , '.dyn_o', '.out' ,'.hi-boot', '.o-boot') # 12112 +def way_supports_io_manager(way: WayName, io_manager: Optional[str]) -> bool: + # All ways support a default IO Manager + if io_manager is None: + return True + + way_is_threaded = '-threaded' in config.way_flags[way] + available_io_managers = config.io_managers_threaded if way_is_threaded else config.io_managers_nonthreaded + return io_manager in available_io_managers + async def test_common_work(name: TestName, opts, func, args) -> None: try: @@ -1621,7 +1656,22 @@ async def test_common_work(name: TestName, opts, # A test itself can request extra ways by setting opts.extra_ways all_ways = list(OrderedDict.fromkeys(all_ways + [way for way in opts.extra_ways if way not in all_ways])) - t.total_test_cases += len(all_ways) + # We split each way over available io managers + if len(opts.io_managers) == 0: + framework_fail(name, None, 'opts.io_managers is empty') + all_ways_and_io_managers = dict( + ( + way, + [io_manager + for io_manager in opts.io_managers + if way_supports_io_manager(way, io_manager) + ] + ) + for way in all_ways + ) + + for io_managers in all_ways_and_io_managers.values(): + t.total_test_cases += len(io_managers) only_ways = getTestOpts().only_ways ok_way = lambda way: \ @@ -1704,19 +1754,23 @@ async def test_common_work(name: TestName, opts, # Run the required tests... for way in do_ways: - if stopping(): - break - start = time.monotonic() - try: - await do_test(name, way, func, args, files) - except KeyboardInterrupt: - stopNow() - except Exception as e: - traceback.print_exc() - framework_fail(name, way, traceback.format_exc(), - runtime=time.monotonic() - start) + for io_manager in all_ways_and_io_managers[way]: + if stopping(): + break + start = time.monotonic() + try: + getTestOpts().io_manager_current = io_manager + await do_test(name ,way ,func ,args ,files) + except KeyboardInterrupt: + stopNow() + except Exception as e: + traceback.print_exc() + framework_fail(name, way, traceback.format_exc(), + runtime=time.monotonic() - start) + + for skipped_way in set(all_ways) - set(do_ways): + t.n_tests_skipped += len(all_ways_and_io_managers[skipped_way]) - t.n_tests_skipped += len(set(all_ways) - set(do_ways)) if getTestOpts().expect == 'missing-lib': t.n_missing_libs += 1 if config.cleanup and do_ways: @@ -1743,6 +1797,8 @@ async def do_test(name: TestName, start = time.monotonic() full_name = name + '(' + way + ')' + if opts.io_manager_current is not None: + full_name += ' (IO Manager: {})'.format(opts.io_manager_current) test_n = len(allTestNames) progress_args = [ full_name, t.total_tests, test_n, [len(t.unexpected_passes), @@ -1925,7 +1981,7 @@ def framework_warn(name: TestName, way: WayName, reason: str) -> None: # run_command. async def run_command( name, way, cmd ): - return await simple_run( name, '', override_options(cmd), '' ) + return await simple_run( name, WayName(''), override_options(cmd), '' ) async def makefile_test( name, way, target=None ): if target is None: @@ -2011,7 +2067,7 @@ async def multi_compile_fail( name, way, top_mod, extra_mods, extra_hc_opts ): return await do_compile( name, way, True, top_mod, extra_mods, [], extra_hc_opts) async def make_depend( name, way, mods, extra_hc_opts ): - return await do_compile( name, way, False, ' '.join(mods), [], [], extra_hc_opts, mode = '-M') + return await do_compile( name, way, False, Path(' '.join(mods)), [], [], extra_hc_opts, mode = '-M') async def do_compile(name: TestName, way: WayName, @@ -2142,7 +2198,7 @@ async def compile_grep_core(name: TestName, async def compile_and_run__(name: TestName, way: WayName, - top_mod: Path, + top_mod: Optional[Path], extra_mods: List[str], extra_hc_opts: str, backpack: bool=False, @@ -2425,6 +2481,9 @@ async def simple_run(name: TestName, way: WayName, prog: str, extra_run_opts: st my_rts_flags = rts_flags(way) + if opts.io_manager_current is not None: + my_rts_flags += ' +RTS --io-manager={0} -RTS '.format(opts.io_manager_current) + # Collect runtime stats if necessary: # isStatsTest and not isCompilerStatsTest(): # assume we are running a ghc compiled program. Collect stats. @@ -2450,6 +2509,12 @@ async def simple_run(name: TestName, way: WayName, prog: str, extra_run_opts: st output_file = perfStatsFile(False, name) + # Check that IO Manager is available + if opts.io_manager_current is not None: + available_io_managers = discover_io_managers(prog, opts.testdir) + if opts.io_manager_current not in available_io_managers: + framework_fail(name, way, 'IO Manager {} unexpectedly unavailable (available IO Managers: {}). `-non-threaded`/`-threaded` is only supported through way RTS flags (not manually added via test modifiers/args) when testing over IO Managers.'.format(opts.io_manager_current, available_io_managers)) + # run the command exit_code = await runCmdPerf(opts.compiler_perf_counters, cmd, output_file, working_dir=opts.testdir, @@ -2511,7 +2576,7 @@ def rts_flags(way: WayName) -> str: async def interpreter_run(name: TestName, way: WayName, extra_hc_opts: str, - top_mod: Path + top_mod: Optional[Path] ) -> PassFail: opts = getTestOpts() @@ -2882,11 +2947,11 @@ async def compare_outputs( if config.accept_platform: if_verbose(1, 'Accepting new output for platform "' + config.platform + '".') - expected_path += '-' + config.platform + expected_path = Path(str(expected_path) + '-' + config.platform) elif config.accept_os: if_verbose(1, 'Accepting new output for os "' + config.os + '".') - expected_path += '-' + config.os + expected_path = Path(str(expected_path) + '-' + config.os) else: if_verbose(1, 'Accepting new output.') @@ -3516,7 +3581,7 @@ def find_expected_file(name: TestName, suff: str, way: WayName) -> Path: for f in files: if in_srcdir(f).exists(): - return f + return Path(f) return basename ===================================== testsuite/driver/testutil.py ===================================== @@ -1,4 +1,5 @@ import os +import re import subprocess import shutil import tempfile @@ -179,3 +180,19 @@ def shorten_metric_name(name: str) -> str: "binary_size/if_compression(3)": "size/3", } return dic.get(name, name) + +# I/O manager discovery for a haskell binary. Returns the set of IO Manager +# strings supported by the binary +def discover_io_managers(hs_exe: Union[str, Path], cwd: Union[str, Path]) -> Set[str]: + rp = subprocess.run( + [str(hs_exe), '+RTS', '--info', '-RTS'], + cwd=cwd, + stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL) + stdout = rp.stdout.decode('utf-8', errors='replace') + + m = re.search(r'$"I/O managers", "(.*)"$', stdout) + if not m: + raise Exception('Expected stdout output of form \'... ("I/O managers", "select selectbis poll") ...\' but got:\n{}'.format(stdout)) + + return set(m.group(1).split()) ===================================== testsuite/tests/rts/all.T ===================================== @@ -674,8 +674,9 @@ test('T23142', [unless(debug_rts(), skip), req_interp], makefile_test, ['T23142' test('T23400', [], compile_and_run, ['-with-rtsopts -A8k']) test('IOManager', [js_skip, when(arch('wasm32'), skip), when(opsys('mingw32'), skip), - pre_cmd('$MAKE -s --no-print-directory IOManager.hs')], - compile_and_run, ['']) + pre_cmd('$MAKE -s --no-print-directory IOManager.hs'), + extra_ways(['threaded2']), all_available_io_managers_except('winio')], + compile_and_run, ['']) test('T24142', [req_target_smp], compile_and_run, ['-threaded -with-rtsopts "-N2"']) ===================================== utils/check-exact/ExactPrint.hs ===================================== @@ -1486,9 +1486,7 @@ markAnnList ann action = do markAnnList' :: (Monad m, Monoid w) => AnnList -> EP w m a -> EP w m (AnnList, a) markAnnList' ann action = do - markAnnListA' ann $ \a -> do - r <- action - return (a,r) + markAnnListA' ann $ action markAnnListA :: (Monad m, Monoid w) => EpAnn AnnList @@ -1503,14 +1501,14 @@ markAnnListA an action = do markAnnListA' :: (Monad m, Monoid w) => AnnList - -> (AnnList -> EP w m (AnnList, a)) + -> EP w m a -> EP w m (AnnList, a) markAnnListA' an action = do an0 <- markLensBracketsO' an lal_brackets an1 <- markEpAnnAllLT' an0 lal_semis - (an2, r) <- action an1 - an3 <- markLensBracketsC' an2 lal_brackets - return (an3, r) + r <- action + an2 <- markLensBracketsC' an1 lal_brackets + return (an2, r) -- --------------------------------------------------------------------- @@ -2413,9 +2411,7 @@ instance ExactPrint (ClsInstDecl GhcPs) where , cid_modifiers = mods }) = do (mbWarn', i', w', mbOverlap', inst_ty', mods') <- top_matter - (ann_list', decls') <- markAnnListA' ann_list $ \a -> do - r <- setLayoutBoth $ mapM markAnnotated decls - return (a,r) + (ann_list', decls') <- markAnnListA' ann_list $ setLayoutBoth $ mapM markAnnotated decls return (ClsInstDecl { cid_ext = (mbWarn', AnnClsInstDecl i' w' ann_list') , cid_poly_ty = inst_ty' , cid_decls = decls' @@ -3336,9 +3332,7 @@ markMaybeDodgyStmts (an,l) stmts = c' <- markEpToken c return (Left (o',c'), r) Right an' -> do - (an'',r') <- markAnnListA' an' $ \a -> do - r <- markAnnotatedWithLayout stmts - return (a, r) + (an'',r') <- markAnnListA' an' $ markAnnotatedWithLayout stmts return (Right an'',r') return ((an0, l), stmts') else return ((an, l), stmts) @@ -3411,9 +3405,7 @@ instance (Typeable body, = MG (origin,an) (L (setAnchorEpa l anc ts cs) matches) exact (MG (origin,an) (L l matches)) = do - (an0,matches') <- markAnnListA' an $ \a -> do - m' <- markAnnotated matches - return (a,m') + (an0,matches') <- markAnnListA' an $ markAnnotated matches return (MG (origin, an0) (L l matches')) -- --------------------------------------------------------------------- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/e3a21c9668a4db6cec6a14fc5c46c6… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/e3a21c9668a4db6cec6a14fc5c46c6… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0

[Git][ghc/ghc][wip/io-manager-deadlock-detection] 39 commits: Add regression test for #7803
by Duncan Coutts (＠dcoutts) 29 Oct '26

29 Oct '26

Duncan Coutts pushed to branch wip/io-manager-deadlock-detection at Glasgow Haskell Compiler / GHC Commits: 234bab08 by Vladislav Zavialov at 2026-09-29T17:22:54-04:00 Add regression test for #7803 Commit 2effe18ab5, first shipped with GHC 8.2.1, fixed #7803 "Superclass methods are left unspecialized". This change turns the example from the ticket description into a regression test, checking that foo1 is fully specialised at Double, with no calls to the overloaded exp5Tail. Closes #7803 Assisted-by: Claude Opus 5 - - - - - c406135c by Andreas Klebinger at 2026-10-01T06:57:28-04:00 RTS: Update flag parsing/handling: Using 0 as argument to any of these flags now resets the flags setting to default: - ``-n0`` disables nursery chunking. - ``-K0`` (or a bare ``-K``) removes the stack size limit. - ``-M0`` removes the heap size limit. - ``-H0`` resets the suggested heap size, including the nursery scaling set by ``-H``. - ``-AL0`` makes the large object allocation limit follow ``-A`` again. GC Generations are now limited to 64. -n larger than -A now emits a warning and disables chunking. Fixes #27830 #27829 - - - - - 984ee82b by Andreas Klebinger at 2026-10-01T06:58:10-04:00 Fix MultiLayerModules timeout multiplier. We accidentally used the compile multiplier for ghci commands which use the run timeout setting to set the timeout. Fixes #26263 and #27796. - - - - - 50a9cf26 by Andreas Klebinger at 2026-10-01T06:58:10-04:00 testsuite: Include runtime in JUnit report. - - - - - e358619f by Alan Zimmerman at 2026-10-01T06:59:01-04:00 EPA: Use EpaLocation in EpVirtualBraces Haddock processing uses EpLayout.EpVirtualBraces to record the current layout column, for allocating haddock comments correctly. For exact printing, we need the line as well as the col, so we know how many lines to skip when the indented code block starts. This commit updates EpLayout.EpVirtualBraces so that it carries an EpaLocation instead of just the Int column, to allow it to be used for both purposes. It is a simple mechanical change, with no other impact, but it allows us to use EpLayout instead of AnnListLayout in AnnList, in the next commit in this series. - - - - - 1d2a26ee by Andrew Lelechenko at 2026-10-01T20:22:04-04:00 Remove bitSize from MINIMAL pragma of class Bits Cf. https://github.com/haskell/core-libraries-committee/issues/441 The function is deprecated for 12 years and has a default implementation, yet it was never removed from the MINIMAL pragma, putting library authors in a very awkward position. Let's finally remove bitSize from MINIMAL. - - - - - 3cc21e4c by Hannes Siebenhandl at 2026-10-01T20:22:59-04:00 Add regression test for #27641 - - - - - c35096f5 by Hannes Siebenhandl at 2026-10-01T20:23:00-04:00 Don't `:add` modules the loader will reject Only home modules can be actually added to the loader context, thus we only accept the module if it belongs to a home unit. - - - - - a563cb27 by Simon Jakobi at 2026-10-02T17:37:36-04:00 ci: accept modified changelog.d entries in lint-changelog The changelog guide says to extend an existing entry rather than add a new one when the feature is already documented. lint_changelog only counted added files, so such MRs failed before the validator ran. Fixes #27779. Assisted-by: Claude Fable 5.1 - - - - - 03082cd4 by Simon Jakobi at 2026-10-02T17:38:17-04:00 changelog.d: Fix warning flag name in the semaphore-v2 entry The entry refers to -Wsemaphore-version-mismatch, which does not exist. The warning GHC emits when it cannot use a -jsem semaphore, including on a protocol version mismatch, is -Wsemaphore-open-failure. Context: !15729, #25087 Assisted-by: Claude Fable 5.1 - - - - - a93705ee by Simon Jakobi at 2026-10-02T17:38:17-04:00 changelog.d: Fix the CLC proposal link in the T27455 entry The entry used the Sphinx :ref: role with a URL as its target. :ref: resolves only internal labels, so the generated release notes would show the proposal name without a link. Use the plain hyperlink form like the other entries. Context: !16274, #27455 Assisted-by: Claude Fable 5.1 - - - - - 54381079 by Hannes Siebenhandl at 2026-10-02T17:39:01-04:00 Extend the `TrustOverlay` for wired-in units The TrustOverlay only contained the `UnitId` of all on-disk units. However, wired-in units are referred to by their wired-in unit id, which is not the same as the on-disk unit id. Thus, trust checks for wired-in units failed because the lookup was using the wired-in unit, which wasn't present in the `TrustOverlay`. We fix #27834, by copying the trust info for wired-in unit ids. - - - - - 11ad6b2f by Duncan Coutts at 2026-10-28T10:46:27+00:00 Move idle GC tracking to its own file, step 1 of 2 Add a new IdleGC.{c,h} module. Move the RecentActivity type, recent_activity variable and {get,set}RecentActivity wrappers. - - - - - 91d4faaa by Duncan Coutts at 2026-10-28T10:46:27+00:00 Move idle GC functionality to its own file, step 2 of 2 We can now fully encapsulate the recent_activity state, exposing just three hooks, and one query used in the scheduler and timer tick. This is a prelude to making changes to the implementation of tracking when to perform an idle gc. - - - - - dc891fa9 by Duncan Coutts at 2026-10-28T10:46:27+00:00 Correct comments in idle GC code The comments still referred to stopTimer/startTimer. These were replaced recently by pauseTimer/unpauseTimer, but the comment was not updated. - - - - - 9d2b0a84 by Duncan Coutts at 2026-10-28T10:46:27+00:00 Make signal handling be a responsibility of the I/O manager(s) Previously it was scattered between I/O managers and the scheduler, and especially the scheduler's deadlock detection. Previously the scheduler would poll for pending signals each iteration of the scheduler loop. The scheduler also had some hairy signal functionality in the deadlock detection: in the non-threaded RTS (only) if there were still no threads running after deadlock detection then it would block waiting for signals. But signals can and (in my opinion) should be thought of as just a funny kind of I/O, and thus should be a responsibility of the I/O manager. So now we have the I/O managers poll for signals when they are polling for I/O completion (and removing the separate poll in the scheduler). And when I/O managers block waiting for I/O then they now also start signal handlers if they get interrupted by a signal. Crucially, if there is no pending I/O or timers, the awaitCompletedTimeoutsOrIO will still block waiting for signals. This patch puts us into an intermediate state: it temporarily breaks deadlock detection in the non-threaded RTS. The waiting on I/O currently happens before deadlock detection. This means we'll now wait forever on signals before doing deadlock detection. We need to move waiting after deadlock detection. We'll do that in a later patch. - - - - - b51b13e2 by Duncan Coutts at 2026-10-28T10:46:27+00:00 Clean up the RTS internal signal handling API Now that the I/O manager is responsible for signals, we can simplify the API we present for signal handling. We now just need startPendingSignalHandlers, which is called from the I/O managers. We can get rid of awaitUserSignals. We also don't need RtsSignals.h to re-export the platform-specific posix/Signals.h or win32/ConsoleHandler.h We can also hide more of the implementation of signals. Less has to be exposed in posix/Signals.h or win32/ConsoleHandler.h. Indeed, posix/Signals.h becomes empty and we remove it. Partly this is because we don't need inline functions (or macros) in the interface. Also remove signal_handlers from RTS ABI exported symbols list. It does not appear to have any users in the core libs, and its really an internal implementation detail. It should not be exposed unless it's really necessary. - - - - - dbea795c by Duncan Coutts at 2026-10-28T12:32:34+00:00 Narrow the window in which signals might be missed before poll/select Any signals delivered just before poll/select block will be missed until they are woken later by unrelated events. This arbitrarily delays the delivery of signals. This is not a complete solution. It is a cheap way to narrow but not close the window. A full solution will have to come later. This issue existed already, and is documented in issue #27540. - - - - - 95520fed by Duncan Coutts at 2026-10-28T12:35:14+00:00 In the scheduler, move I/O blocking after deadlock detection To make deadlock detection effective in the non-threaded RTS when there are deadlocked threads and other unrelated threads waiting on I/O, we need to arrange to do deadlock detection before we block in scheduler to wait on I/O. The solution is to: 1. adjust scheduleFindWork, which runs before deadlock detection, to only poll for I/O and not block; and 2. add a step after deadlock detection to wait on I/O if there are still no threads to run (and there's any I/O or timeouts outstanding) The scheduleCheckBlockedThreads is now so simple that it made more sense to inline it into scheduleFindWork. - - - - - 7e8c0432 by Duncan Coutts at 2026-10-28T12:35:14+00:00 Remove bogus anyPendingTimeoutsOrIO guard from scheduleDetectDeadlock The deadlock detection was only invoked if both of these conditions hold: 1. the run queue is empty 2. there is no pending I/O or timeouts The second condition is unnecessary. The deadlock detection mechanism can find deadlocks even if there are other threads waiting on I/O or timers. Having this extra condition means that we fail to detect blocked threads if there are any threads waiting on I/O or timers. Part of fixing issue #26408 - - - - - ce1f7e26 by Duncan Coutts at 2026-10-28T12:35:14+00:00 Don't consider pending I/O for early context switch optimisation Context switches are normally initiated by the timer signal. If however the user specifies "context switch as often as possible", with +RTS -C0 then the scheduler arranges for an early context switch (when it's just about to run a Haskell thread). Context switching very often is expensive, so as an optimisation there cases where we do not arrange an early context switch: 1. if there's no other threads to run 2. if there is no pending I/O or timers This patch eliminates case 2, leaving only case 1. The rationale is as follows. The use of this was inconsistent across platforms and threaded/non-threaded RTS ways. It only worked on the non-threaded RTS and on Windows only worked for the win32-legacy I/O manager. On all other combinations anyPendingTimeoutsOrIO would always return false. The fact that nobody noticed and complained about this inconsistency suggests that the feature is not relied upon. If however it turns out that applications do rely on this, then the proper thing to do is not to restore this check, but to add a new I/O manager hint function that returns if there is any pending events that are likely to happen *soon*: for example timeouts expiring within one timeslice, or I/O waits on things likely to complete soon like disk I/O, but not for example socket/pipe I/O. The motivation to avoid this use of anyPendingTimeoutsOrIO is to allow us to eliminate anyPendingTimeoutsOrIO entirely. All other uses of this are just guards on {await,poll}CompletedTimeoutsOrIO and the guards can safely be folded into those functions. This will better cope with some I/O managers having no proper implementation of anyPendingTimeoutsOrIO. Ultimately this will let us simplify the scheduler which currently has to have special #ifdef mingw32_HOST_OS cases to cope with the lack of a working anyPendingTimeoutsOrIO for some Windows I/O managers - - - - - e7c7595e by Duncan Coutts at 2026-10-28T12:35:14+00:00 Remove anyPendingTimeoutsOrIO guarding {poll,await}CompletedTimeoutsOrIO Previously the API of the I/O manager used a two step process: check anyPendingTimeoutsOrIO and then call {poll,await}CompletedTimeoutsOrIO. This was primarily there as a performance thing, to cheaply check if we need to do anything. And then because anyPendingTimeoutsOrIO existed, it was used for other things too. We have now eliminated the other uses, and are just left with the performance pattern. But this was problematic because not all I/O managers correctly implement anyPendingTimeoutsOrIO (specifically the win32 ones), and now that we also make I/O managers responsible for signals then we need to poll/await even if there is no pending I/O or timeouts. If there is no pending I/O or timeouts then await needs to degenerate to just waiting forever for any signals. - - - - - a8675b65 by Duncan Coutts at 2026-10-28T12:35:14+00:00 Remove anyPendingTimeoutsOrIO, it is no longer used And this avoids the problems arising from the win32 I/O managers having had a bogus implementation. - - - - - 6665b6fa by Duncan Coutts at 2026-10-28T12:35:15+00:00 Remove second scheduler call to awaitCompletedTimeoutsOrIO Previously awaitCompletedTimeoutsOrIO was called both before and after deadlock detection in the scheduler. The reason for that was that the win32 I/O managers had a bogus implementation of anyPendingTimeoutsOrIO and this was used to guard the call of awaitCompletedTimeoutsOrIO prior to deadlock detection. This meant the first call site was never actually called when using the win32 I/O managers. This was the reason for the second call: the first one was never used. What a mess. So now we have a simple design in the scheduler: 1. poll for completed I/O, timers or signals 2. if no runnable threads: do deadlock detection 3. if still no runnable threads: block waiting for I/O, timers or signals. - - - - - 5b2ba4f3 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Lift emptyRunQueue guard out of scheduleDetectDeadlock this improved the clarity of the logic when reading the scheduler code. And improve the comment about scheduleDoGC that was rather unclear. - - - - - b6158704 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Make interruptIOManager work for WinIO (non-threaded) Fixes #27403 The idle GC scheme now fundamentally relies on interruptIOManager working, so it's no longer acceptable for it to be missing for WinIO. Not having it would cause lock ups. - - - - - fc133aab by Duncan Coutts at 2026-10-28T12:35:15+00:00 Make non-threaded deadlock detection also rely on idle GC Only do deadlock detection GC when idle GC kicks in. This also relies on using wakeUpRts, so now do this unconditionally. Previously wakeUpRts was for the threaded rts only. - - - - - 713f3594 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Enable idle GC by default on non-threaded RTS The behaviour is now uniform between the threaded and non-threaded RTS ways. The deadlock detection now relies on idle GC for both threaded and non-threaded ways. Previously deadlock detection did not rely on idle GC for the non-threaded way. Also tweak test T7275 to account for idle GC. This test's output is sensitive to the number of major GCs run. Since this commit enables idle GC for the non-threaded RTS, for this test that increases the number of major GCs, since the test program is frequently idle for more than 300ms. - - - - - da537110 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Fix state of idle GC control vars with +RTS -V0 Currently when the user uses +RTS -I0, then doIdleGC is set to false. But if the master tick interval -V is set to 0 then the idleGCDelayTime was being set to 0 but doIdleGC was not being set to false, which is inconsistent, and almost certainly buggy. - - - - - 656ea7e0 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Add a long Note [Deadlock detection] It describes the historical and modern designs and their trade-offs. The point is we've now unified the code for deadlock detection between the threaded and non-threaded ways, by changing the non-threaded to follow the same design as the threaded. - - - - - 93c6c1de by Duncan Coutts at 2026-10-28T12:35:15+00:00 Refactor the utils providing the I/O timeout for different system APIs We have to provide a timeout in three different time formats (int milliseconds, struct timeval *, and struct timespec *). These are for platform APIs including poll, select, ppoll, pselect, epoll_wait, epoll_wait2 and in future kevent and io_uring. Previously we had one util for each format that both worked out what the timeout should be and converted it into the particular format. This duplicates some logic and it also makes it awkward to modify the timeout value (since there's no common format). Modifying the timeout value is exactly what we want to do, in a subsequent commit, where we want to adjust the timeout to take account of idle GC. - - - - - bc5206b0 by Duncan Coutts at 2026-10-28T12:35:15+00:00 Update the old select I/O manager to use the new timeout utils This will make it much easier to adjust the timeout to take account of idle GC. - - - - - 07d7ce9c by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add opportunistic deadlock detection without idle GC Our general design is now that comprehensive deadlock detection relies on idle GC. This allows us to detect partial deadlocks that are not system deadlocks. But it's not prompt. It's a nice to have feature to be able to promptly detect the cases that are system deadlocks. There's no hope to do this in the threaded RTS with -N > 1, but it can be done for the non-threaded RTS and _could_ be done for the threaded RTS with -N1. We can integrate this feature into the general design without too many contortions. We just need the I/O managers to check before sleeping if they would be about to sleep indefinately with no I/O and no registered signal handlers. If so, they can notify the idle GC that we're deadlocked and we schedule and idle GC. - - - - - f47f7c22 by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add idle GC mode for !HAVE_PREEMPTION We have to do idle GC (and thus deadlock detection) differently when we do not have the ticker and the ability to interrupt the I/O manager when it's blocking. - - - - - 7a40abb2 by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add and use USED_IF_PREEMPTION to avoid warnings - - - - - a33b2aae by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add performDeadlockDetection and export via ghc-experimental When documenting the idle GC and deadlock detection behaviour in the user guide, it becomes obiovus that in some cases one would want manual control over deadlock detection. In particular, for applications that disable idle GC, for debugging or robustness, they may want to invoke deadlock detecting GC manually. Bind it in ghc-internal GHC.Internal.System.Mem and re-export via ghc-experimental in System.Mem.Experimental. It is clearly implementation specific behaviour, especially since in GHC the deadlock detection is tied to doing a major GC. - - - - - 8e89ffbc by Duncan Coutts at 2026-10-29T10:51:00+00:00 Add tests for deadlock detection, issue #26408 We have three tests: T26408a: make sure that we can detect a /partial deadlock/ that is not also a /system deadlock/ T26408b: make sure that we can detect a /system deadlock/ promptly without relying on idle GC T26408c: make sure that we do not falsely detect a deadlock when there is nothing to do except wait on signals T26408d: make sure that we can detect a system deadlock with idle GC disabled, after the idle GC timeout has elapsed. - - - - - 5baa4651 by Duncan Coutts at 2026-10-29T10:51:00+00:00 Update the user guide with the revised idle GC behaviour It's now not just for the threaded RTS, but general. Add a new section in the RTS chapter: Idle GC and deadlock detection. Provide general documentation on both topics. In particular, document the behaviour of deadlock detection and how it depends on idle GC. Move the existing entries for -I and -Iw to this section. Add a changelog entry. - - - - - fed3354f by Duncan Coutts at 2026-10-29T10:51:00+00:00 Remove variable n_haskell_handlers which has become write-only - - - - - 108 changed files: - .gitlab/ci.sh - + changelog.d/27641 - changelog.d/T27455 - + changelog.d/T27830-rts-flag-bounds - + changelog.d/idle-gc-and-deadlock-detection - changelog.d/semaphore-v2 - changelog.d/unit-index - compiler/GHC/Parser.y - compiler/GHC/Parser/Annotation.hs - compiler/GHC/Parser/PostProcess/Haddock.hs - compiler/GHC/Unit/External/Wired.hs - compiler/GHC/Unit/State.hs - docs/users_guide/profiling.rst - docs/users_guide/runtime_control.rst - ghc/GHCi/UI.hs - libraries/base/changelog.md - libraries/ghc-experimental/src/System/Mem/Experimental.hs - libraries/ghc-internal/src/GHC/Internal/Bits.hs - libraries/ghc-internal/src/GHC/Internal/System/Mem.hs - rts/Capability.c - rts/IOManager.c - rts/IOManager.h - + rts/IdleGC.c - + rts/IdleGC.h - rts/Linker.c - rts/RtsFlags.c - rts/RtsSignals.h - rts/RtsStartup.c - rts/RtsSymbols.c - rts/Schedule.c - rts/Schedule.h - rts/Timer.c - rts/include/rts/Flags.h - rts/include/rts/OSThreads.h - rts/include/rts/storage/GC.h - rts/posix/Poll.c - rts/posix/Poll.h - rts/posix/Select.c - rts/posix/SelectBis.c - rts/posix/SelectBis.h - rts/posix/Signals.c - − rts/posix/Signals.h - rts/posix/Timeout.c - rts/posix/Timeout.h - rts/rts.cabal - rts/sm/GC.c - rts/sm/Storage.c - rts/win32/AsyncMIO.c - rts/win32/AsyncMIO.h - rts/win32/AsyncWinIO.c - rts/win32/AsyncWinIO.h - rts/win32/AwaitEvent.c - rts/win32/ConsoleHandler.c - rts/win32/ConsoleHandler.h - testsuite/driver/junit.py - testsuite/driver/testglobals.py - testsuite/driver/testlib.py - testsuite/tests/ghc-api/exactprint/T22919.stderr - testsuite/tests/ghc-api/exactprint/Test20239.stderr - testsuite/tests/ghc-api/exactprint/ZeroWidthSemi.stderr - + testsuite/tests/ghci/scripts/T27641.script - + testsuite/tests/ghci/scripts/T27641.stderr - + testsuite/tests/ghci/scripts/T27641.stdout - + testsuite/tests/ghci/scripts/T27834.hs - + testsuite/tests/ghci/scripts/T27834.script - + testsuite/tests/ghci/scripts/T27834.stdout - testsuite/tests/ghci/scripts/all.T - testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T17544_kw.stderr - testsuite/tests/haddock/should_compile_flag_haddock/T24221.stderr - testsuite/tests/interface-stability/base-exports.stdout - testsuite/tests/interface-stability/base-exports.stdout-javascript-unknown-ghcjs - testsuite/tests/interface-stability/base-exports.stdout-mingw32 - testsuite/tests/interface-stability/ghc-experimental-exports.stdout - testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32 - testsuite/tests/module/mod185.stderr - testsuite/tests/parser/should_compile/DumpParsedAst.stderr - testsuite/tests/parser/should_compile/DumpParsedAstComments.stderr - testsuite/tests/parser/should_compile/DumpSemis.stderr - testsuite/tests/parser/should_compile/KindSigs.stderr - testsuite/tests/parser/should_compile/T15323.stderr - testsuite/tests/parser/should_compile/T20452.stderr - testsuite/tests/parser/should_compile/T20718.stderr - testsuite/tests/parser/should_compile/T20718b.stderr - testsuite/tests/parser/should_compile/T20846.stderr - testsuite/tests/parser/should_compile/T23315/T23315.stderr - testsuite/tests/perf/compiler/all.T - testsuite/tests/printer/AnnotationNoListTuplePuns.stdout - testsuite/tests/printer/T18791.stderr - testsuite/tests/printer/Test20297.stdout - testsuite/tests/printer/Test24533.stdout - testsuite/tests/profiling/should_run/Makefile - testsuite/tests/rts/Makefile - + testsuite/tests/rts/T26408a.hs - + testsuite/tests/rts/T26408a.stderr - + testsuite/tests/rts/T26408b.hs - + testsuite/tests/rts/T26408b.stderr - + testsuite/tests/rts/T26408c.hs - + testsuite/tests/rts/T26408c.stdout - + testsuite/tests/rts/T26408d.hs - + testsuite/tests/rts/T26408d.stderr - testsuite/tests/rts/all.T - testsuite/tests/rts/rtsflags001.stderr-ws-32 - testsuite/tests/rts/rtsflags001.stderr-ws-64 - + testsuite/tests/simplCore/should_compile/T7803.hs - + testsuite/tests/simplCore/should_compile/T7803.stderr - + testsuite/tests/simplCore/should_compile/T7803a.hs - testsuite/tests/simplCore/should_compile/all.T The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/04c391127b3843dbc5f33550b96264… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/04c391127b3843dbc5f33550b96264… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0

[Git][ghc/ghc][wip/T27994] Fix a nasty bug in fundeps
by Simon Peyton Jones (＠simonpj) 29 Oct '26

29 Oct '26

Simon Peyton Jones pushed to branch wip/T27994 at Glasgow Haskell Compiler / GHC Commits: 565f8040 by Simon Peyton Jones at 2026-10-29T09:24:16+00:00 Fix a nasty bug in fundeps This MR fixes #27994, which turned out to be very tricky. See Note [Exploiting closed type famililes], especially (CF7). I was ending up with TyVars (not TcTyVars) in the solver, because the substitution computed in `getRelevantBranches` mixed the two. The code was just too clever. I ended up simplifying it quite a bit, eliminating fancy footwork in a function `trim_qtvs`. I left behind Historical Note [trim_qtvs] to explain. This patch also fixes one other buglet in the same code. * Over-saturated type families were being mis-handled. See the new use of `saturatedFamApp_maybe`. - - - - - 7 changed files: - compiler/GHC/Core/Coercion/Axiom.hs - compiler/GHC/Core/Predicate.hs - compiler/GHC/Tc/Instance/Family.hs - compiler/GHC/Tc/Solver/FunDeps.hs - + testsuite/tests/typecheck/should_compile/T27994.hs - + testsuite/tests/typecheck/should_compile/T27994a.hs - testsuite/tests/typecheck/should_compile/all.T Changes: ===================================== compiler/GHC/Core/Coercion/Axiom.hs ===================================== @@ -277,8 +277,8 @@ data CoAxBranch -- See Note [CoAxBranch type variables] , cab_eta_tvs :: [TyVar] -- ^ Eta-reduced tyvars - -- cab_tvs and cab_lhs may be eta-reduced; see - -- Note [Eta reduction for data families] + -- For a data family (only) cab_tvs and cab_lhs may be eta-reduced + -- See Note [Eta reduction for data families] , cab_cvs :: [CoVar] -- ^ Bound coercion variables -- Always empty, for now. @@ -286,8 +286,14 @@ data CoAxBranch -- in GHC.Tc.TyCl , cab_roles :: [Role] -- ^ See Note [CoAxBranch roles] + , cab_lhs :: [Type] -- ^ Type patterns to match against + -- For a data family `cab_lhs` may be shorter than the arity + -- of the data type + -- For a type family `cab_lhs` is exactly the family arity + -- See Note [Eta reduction for data families] + , cab_rhs :: Type -- ^ Right-hand side of the equality -- See Note [CoAxioms are homogeneous] ===================================== compiler/GHC/Core/Predicate.hs ===================================== @@ -36,7 +36,7 @@ module GHC.Core.Predicate ( -- Equality left-hand sides CanEqLHS(..), canEqLHS_maybe, canTyFamEqLHS_maybe, - canEqLHSKind, canEqLHSType, eqCanEqLHS + canEqLHSKind, canEqLHSType, eqCanEqLHS, saturatedFamApp_maybe ) where @@ -61,6 +61,8 @@ import GHC.Utils.Misc import GHC.Utils.Panic import GHC.Data.FastString +import GHC.Exts( inline ) + {- ********************************************************************* * * @@ -700,7 +702,9 @@ types/kinds are fully settled and zonked. -- See also Note [Ordering of implicit variables] in "GHC.Rename.HsType" scopedSort :: [Var] -> [Var] -scopedSort = go [] [] +scopedSort xs + | null xs = [] + | otherwise = go [] [] xs where go :: [Var] -- already sorted, in reverse order -> [TyCoVarSet] -- each set contains all the variables which must be placed @@ -781,13 +785,19 @@ canEqLHS_maybe xi | otherwise = canTyFamEqLHS_maybe xi -canTyFamEqLHS_maybe :: Type -> Maybe CanEqLHS -canTyFamEqLHS_maybe xi +saturatedFamApp_maybe :: Type -> Maybe (TyCon,[Type]) +saturatedFamApp_maybe xi | Just (tc, args) <- tcSplitTyConApp_maybe xi , isTypeFamilyTyCon tc , args `lengthIs` tyConArity tc - = Just $ TyFamLHS tc args + = Just (tc,args) + | otherwise + = Nothing +canTyFamEqLHS_maybe :: Type -> Maybe CanEqLHS +canTyFamEqLHS_maybe xi + | Just (tc, args) <- inline saturatedFamApp_maybe xi + = Just $ TyFamLHS tc args | otherwise = Nothing ===================================== compiler/GHC/Tc/Instance/Family.hs ===================================== @@ -16,6 +16,7 @@ import GHC.Prelude import GHC.Driver.DynFlags import GHC.Driver.Env +import GHC.Core.Predicate( saturatedFamApp_maybe ) import GHC.Core.FamInstEnv import GHC.Core.Coercion import GHC.Core.TyCon @@ -715,15 +716,9 @@ reportInjectivityErrors dflags fi_ax axiom inj when wrong_bare_rhs $ reportBareVariableInRHSErr fam_tc bare_variables axiom -- | Is type headed by a type family application? -isTFHeaded :: Type -> Bool -- See Note [Verifying injectivity annotation], case 3. -isTFHeaded ty | Just ty' <- coreView ty - = isTFHeaded ty' -isTFHeaded ty | (TyConApp tc args) <- ty - , isTypeFamilyTyCon tc - = args `lengthIs` tyConArity tc -isTFHeaded _ = False - +isTFHeaded :: Type -> Bool +isTFHeaded ty = isJust (saturatedFamApp_maybe ty) -- | If a RHS is a bare type variable return a set of LHS patterns that are not -- bare type variables. ===================================== compiler/GHC/Tc/Solver/FunDeps.hs ===================================== @@ -24,16 +24,16 @@ import GHC.Tc.Types.Constraint import GHC.Core.FamInstEnv import GHC.Core.Coercion import GHC.Core.TyCo.Rep( Type(..) ) -import GHC.Core.Predicate( EqRel(..) ) +import GHC.Core.TyCo.Subst( deepSubstTy, elemSubst ) +import GHC.Core.Predicate( EqRel(..), saturatedFamApp_maybe ) import GHC.Core.TyCon import GHC.Core.Unify( AmIUnifying(..) , tcUnifyTysForInjectivity, typeListsAreApart ) import GHC.Core.Coercion.Axiom -import GHC.Core.TyCo.Subst( elemSubst, extendTvSubstWithClone ) import GHC.Builtin.WiredIn.TypeLits( tryInteractTopFam, tryInteractInertFam ) -import GHC.Types.Var( TyVar, tyVarKind, setTyVarKind ) +import GHC.Types.Var( tyVarKind ) import GHC.Types.Var.Set import GHC.Types.Var.Env( mkInScopeSet ) @@ -57,7 +57,6 @@ Here is our plan for dealing with functional dependencies * send the current constraint back to the start of the pipeline; it might now be soluble, and it probably isn't inert - * Once a unification is done, it can't be un-done. So we must be very careful not to threaten completeness by doing a unification when there might be another way to solve the constraint. @@ -613,6 +612,7 @@ tryFamEqFunDeps mode eqs_for_me fam_tc work_args mkTopFamEqFDs :: TyCon -> [TcType] -> EqCt -> SolverStage [FunDepEqns] mkTopFamEqFDs fam_tc work_args work_item + -- NB: work_args exactly saturate fam_tc | isOpenTypeFamilyTyCon fam_tc , Injective inj_flags <- tyConInjectivityInfo fam_tc = -- Open, injective type families @@ -709,24 +709,24 @@ getRelevantBranches ax work_args work_rhs , cab_lhs = lhs_tys , cab_rhs = rhs_ty }) | Just subst <- tcUnifyTysForInjectivity Unifying (rhs_ty:lhs_tys) work_tys - -- See (RW1)-(RW3) in Note [Relevance] - -- This is a pre-/unification/ check (not matching) - -- so `subst` can bind both `qtvs` and fvs(work_tys) - , let (subst',qtvs') = trim_qtvs subst qtvs - lhs_tys' = substTys subst' lhs_tys - rhs_ty' = substTy subst' rhs_ty - branch_tys' = rhs_ty' : lhs_tys' + -- See (RW1)-(RW3) in Note [Relevance] + , let lhs_tys' = map (deepSubstTy subst) lhs_tys + -- deepSubstTy: see (CF8) in Note [Exploiting closed type families] , noEarlierMatchCheck lhs_tys' branch -- See (RW4) in Note [Relevance] - = Just (FDEqns { fd_qtvs = qtvs' - , fd_eqs = zipWith Pair branch_tys' work_tys }) - -- Do not apply the substition to `work_tys`; - -- See (CF7) in Note [Exploiting closed type families] + = -- Ignore the substitution, except for `noEarlierMatchCheck` + -- See (CF7) in Note [Exploiting closed type families] + Just (FDEqns { fd_qtvs = qtvs + , fd_eqs = zipWith Pair (rhs_ty:lhs_tys) work_tys }) + -- Do not apply the substitution to `work_tys`; + -- See (CF9) in Note [Exploiting closed type families] + | otherwise = Nothing mkTopOpenFamEqFDs :: TyCon -> [Bool] -> [TcType] -> EqCt -> TcS [FunDepEqns] -- Implements (INJFAM:Wanted/top-open) -- See Note [Exploiting open type families] +-- NB: work_args exactly saturate fam_tc mkTopOpenFamEqFDs fam_tc inj_flags work_args (EqCt { eq_rhs = work_rhs }) = do { fam_envs <- getFamInstEnvs ; let branches :: [CoAxBranch] @@ -739,11 +739,17 @@ mkTopOpenFamEqFDs fam_tc inj_flags work_args (EqCt { eq_rhs = work_rhs }) , cab_lhs = branch_lhs_tys , cab_rhs = branch_rhs }) | Just subst <- tcUnifyTysForInjectivity Matching [branch_rhs] [work_rhs] - , let (subst', qtvs') = trim_qtvs subst branch_qtvs - branch_lhs_tys' = substTys subst' branch_lhs_tys - -- No need for an apartness check here; for open type families - -- there are no "preceding" equations - = Just (mkInjectivityFDEqn inj_flags qtvs' branch_lhs_tys' work_args) + -- No need for an apartness check here; + -- for open type families there are no "preceding" equations + -- Both branch_lhs_tys and work_args exactly saturate fam_tc + = Just $ + if all (`elemSubst` subst) branch_qtvs -- See (OTF1) + then mkInjectivityFDEqn [] inj_flags + (substTys subst branch_lhs_tys) + work_args + else mkInjectivityFDEqn branch_qtvs (True: inj_flags) + (branch_rhs: branch_lhs_tys) + (work_rhs: work_args) | otherwise = Nothing @@ -756,9 +762,9 @@ mkLocalFamEqFDs eqs_for_me fam_tc inj_flags work_args work_rhs = do { let -- eqns_from_inerts: see (INJFAM:Wanted/other) eqns_from_inerts = mapMaybe do_one eqs_for_me -- eqns_from_self: see (INJFAM:Wanted/Self) - eqns_from_self = case tcSplitTyConApp_maybe work_rhs of - Just (tc,rhs_tys) | tc==fam_tc -> [mk_eqn rhs_tys] - _ -> [] + eqns_from_self = case saturatedFamApp_maybe work_rhs of + Just (tc, rhs_tys) | fam_tc == tc -> [mk_eqn rhs_tys] + _ -> [] ; return (eqns_from_inerts ++ eqns_from_self) } where @@ -768,7 +774,10 @@ mkLocalFamEqFDs eqs_for_me fam_tc inj_flags work_args work_rhs | otherwise = Nothing do_one _ = pprPanic "interactFunEq 2" (ppr fam_tc) -- TyVarLHS - mk_eqn iargs = mkInjectivityFDEqn inj_flags [] work_args iargs + mk_eqn iargs = mkInjectivityFDEqn [] inj_flags work_args iargs + + +{- See Historical Note [trim_qtvs] trim_qtvs :: Subst -> [TyVar] -> (Subst, [TyVar]) -- Tricky stuff: see (OF1) in Note [Exploiting open type families] @@ -790,7 +799,7 @@ trim_qtvs unif_subst qtvs subst' = extendTvSubstWithClone subst tv tv' -- extendTvSubstWithClone: see (OF1b) in -- Note [Exploiting open type families] - +-} ----------------------------------------- -- Built-in type families @@ -883,15 +892,22 @@ mkLocalBuiltinFamEqFDs eqs_for_me fam_tc ops work_args work_rhs ; return (eqns_from_inerts ++ eqns_from_self) } -------------------- -mkInjectivityFDEqn :: [Bool] -- Injectivity flags - -> [TcTyVar] -- Quantify these +mkInjectivityFDEqn :: [TcTyVar] -- Quantify these + -> [Bool] -- Injectivity flags -> [TcType] -> [TcType] -- Make these equal -> FunDepEqns -- When F s1 s2 s3 ~ F t1 t2 t3, and F has injectivity info [True,False,True] -- return the FDEqns { fd_eqs = [Pair s1 t1, Pair s3 t3] } --- The injectivity flags [Bool] will not all be False, but nothing goes wrong if they are -mkInjectivityFDEqn inj_args qtvs lhs_args rhs_args - = FDEqns { fd_qtvs = qtvs, fd_eqs = eqs } +-- The injectivity flags [Bool] will not all be False, +-- but nothing goes wrong if they are +mkInjectivityFDEqn qtvs inj_args lhs_args rhs_args + = assertPpr (let len = length inj_args in + len == length lhs_args && len == length rhs_args) + (vcat [ text "inj_args:" <+> ppr inj_args + , text "lhs_args:" <+> ppr lhs_args + , text "rhs_args:" <+> ppr rhs_args ]) $ + -- Need inj_args lhs_args and rhs_args to be same length for zip3 + FDEqns { fd_qtvs = qtvs, fd_eqs = eqs } where eqs = [ Pair lhs_arg rhs_arg | (True, lhs_arg, rhs_arg) <- zip3 inj_args lhs_args rhs_args ] @@ -1104,7 +1120,7 @@ the Injective Type Families paper. For open type families, suppose we have [W] F alpha ~ Int->Int -Then, since F has been checked as injective, and the RHS `a->a` (pre-)unifies +Then, since F has been checked as injective, and the RHS `a->a` (pre-)matches with the `Int->Int` in the Wanted with substitution [a :-> Int], we can add a derived FunDepEqns FDEqns{ fd_qtvs = [], fd_eqs = [ [Int] ~ alpha ] } @@ -1119,7 +1135,7 @@ instance are bound by the pre-unification step. Suppose we have [W] F alpha ~ [ty] -Then, the RHS [G a] pre-unifies with the RHS of the Wanted, but with an +Then, the RHS [G a] pre-matches with the RHS of the Wanted, but with an empty substitution, which does not fix `a`. So we get this FunDepEqns: FDEqns{ fd_qtvs = a, fd_eqs = [ Maybe a ~ alpha ] } (note the fd_qtvs=a) and thereby emit (via `instantiateFunDepEqns`) @@ -1127,42 +1143,65 @@ empty substitution, which does not fix `a`. So we get this FunDepEqns: We have learned that `alpha` must be a list, but that is all; we don't know what is inside it, hence the fresh `gamma`. -In general we have: - - type family F x1 ... xn = r | r -> x1 x3 (say) - type instance forall qs. F t1 ... tn = rhs - [W] F a1 ... an ~ b - -Suppose `b` pre-unifies with `rhs`, with substitution `S` that binds -some (but perhaps not all) of the `qs`. Lets say - unbound_qs = qs \ dom(S) - -Then we generate the FunDepEqs - FDEqns{ fd_qtvs = unbound_qs - , fd_eqs = [ S(t1) ~ S(a1), S(t3) ~ S(a3) ] - -These 'fd_qtvs' are subsequently instantiated to fresh metavariables by -'instantiateFunDepEqns'. - -(OF1) The computation of `unbound_qs` is done by `trim_qtvs`, but we - have to be careful. Suppose (OF1a) - qs = [ k::*, a::k ] - S = [ k :-> UnliftedType ] - Then we want - unbound_qs = [ a::UnliftedType ] -- Must apply S to q's kind!! - - And (OF1b), although I'm not sure if this can acutally happen: - qs = [ k::*, a::(k::*), b::(a::k) ] - S = [ k :-> UnliftedType ] - We want - unbound_qs = [ a::UnliftedType, b::(a::UnliftedType) ] - Note that the occurrence of `a` in b's kind should be replaced by the - newly-kinded `a` not the original `a::k`. - - This fancy footwork for `fd_qtvs` is done by `trim_qtvs`. It - * Drops a quantified tyvar that /is/ matched - * Substitutes in the kind of a quantified tyvar `a` that is /not/ matched - * Adds the substitution [a::k :-> a::S(k)] to the substitution +It gets tricker. Suppose we have + type family F @k a = r | r -> k,a + type instance F @k (Proxy @k (a::k)) = T k + [W] F alpha beta ~ T Type +Matching the RHS gives substitution [k :-> Type]. But we do /not/ want +to end up with this: + FDEqns{ fd_qtvs = (a::k), fd_eqs = [ Proxy @Type (a::k) ~ alpha ] } +becuase that `k`, hidden inside the kind of `a`, escapes into the solver. + +Previously we tried fancy footwork -- see Historical Note [trim_qtvs]. +But now we do the following, given + type family F a b = r | r -> b + type instance F lhs1 lhs2 = rhs + [W] F wlhs1 wlhs2 ~ wrhs + +* We can safely abandon the sustitution entirely and generate + FDEqns{ fd_qtvs = [a,b], fd_eqs = [ lhs2 ~ wlhs2, rhs ~ wrhs ] } + By generating (rhs ~ wrhs) we effectively make the solver re-generate the + substitution. This is also what we do for closed type families; see (CF7). + These 'fd_qtvs' are subsequently instantiated to fresh meta-variables by + 'instantiateFunDepEqns'. See Historical Note [trim_qtvs]. + +* As a fast-path optimisation, if the substitution (a /matching/ substition only, + remember) covers all the `qtvs` of the type instance, there's no hidden-kind problem. + Just apply the substitution and equate, with fd_qtvs = []: + FDEqns{ fd_qtvs = [], fd_eqs = [ S(lhs2) ~ wlhs2 } + +Historical Note [trim_qtvs] +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +In Note [Exploiting open type families], we previously tried to make use of +the matching substitution `S`, by defining + unbound_qs = qs \ dom(S) +so that we could make fresh meta-variables for only the unbound `qs`. +But it turned out to be tricky. Suppose (TQ1) + qs = [ k::*, a::k ] + S = [ k :-> UnliftedType ] +Then we want + unbound_qs = [ a::UnliftedType ] -- Must apply S to q's kind!! + +And (TQ2), although I'm not sure if this can actually happen: + qs = [ k::*, a::(k::*), b::(a::k) ] + S = [ k :-> UnliftedType ] +We want + unbound_qs = [ a::UnliftedType, b::(a::UnliftedType) ] +Note that the occurrence of `a` in b's kind should be replaced by the +newly-kinded `a` not the original `a::k`. + +This fancy footwork for `fd_qtvs` used to be done by `trim_qtvs`. It + * Drops a quantified tyvar that /is/ matched + * Substitutes in the kind of a quantified tyvar `a` that is /not/ matched + * Adds the substitution [a::k :-> a::S(k)] to the substitution + +(TQ3) This was hard enough for open type type families, which use matching. +But it was ridiculously hard for /closed/ type families which use pre-/unification/ +so that the substitution `S` has both meta-tyvars and instance-head tyvars in +the range. + +It's much easier to drop all this and just make fresh meta-tyvars for +all the quantified type variables. Note [Exploiting closed type families] see `mkTopClosedFamEqFDs` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1194,9 +1233,6 @@ More precisely, here is the very clever Closed Family Fundep Algorithm (CFFA) with fresh unification vars in lhs and rhs for the quantified variables of the equation. - Actually we take advantage of the substitution generated by the relevance - check, see (CF7) below. - See Note [Relevance] for what "relevant" means. We need to take care about non-termination; see (CF3). @@ -1391,26 +1427,59 @@ Key point: equations that are not relevant do not need to be considered for fund (CF7) Relevance. When computing relevance in the Closed Family Fundep Algorithm, for [W] F wlhs1 wlhs2 ~ wrhs, against an equation F lhs1 lhs2 ~ rhs we get back a pre-unifying substitution S that makes the two as equal as poss. - Then we use `trim_qtvs` in exactly the same way as in - (OF1) of Note [Exploiting open type families], to generate qtvs' and S' - Then we generate - FDEqns { fd_qtvs = qtvs', fd_eqs [ S'(lhs1) ~ wlhs1 - , S'(lhs2) ~ wlhs2 - , S'(rhs) ~ wrhs ] } - Notice that we do /not/ apply S' to wlhs/wrhs. These type contain unification - variables that we must actually unify! Example + This substitution can bind both /quantified/ type variables from the instance, + and /unification variables/ from the solver. Moreover, both kind of type variable + can appear in the range of the substitution. + + That's pretty complicated, so we are very cautious about using the returned + substitution. I had a long struggle (#27994) when we were previously trying to + apply that returned substitution as an optimisation. But because type variables + can occur in kinds, it was a losing battle -- I ended up with TyVars from the + instance head escaping into the solver. Consider + + type F :: forall k. k -> * + type instance F k a where + F (k->*) (b::k->*) = Int + + [W] F kappa (alpha::kappa) ~ Int + + We unify the Wanted with the equation to get the substitution + [ kappa :-> k->*, b :-> alpha ] + I tried using `trim_qtvs` (see Historical Note [trim_qtvs]) but my head hurt + FAR too much: #27994 is an excellent example. I gave up and now just ignore + the substitution, making a fresh meta-var for each quantified variable. + +(CF8) When computing relevance (Note [Relevance] (R2)) we use `noEarlierMatch` + on S(lhs). But we must use /deep/ substitution on LHS. Consider + type family F (a :: k) r where + F @Type (Maybe Bool) r = r -- B0 + F @Type ((f :: Type -> Type) Int) r = Bool -- B1 + F @k ((g :: Type -> k) Int) r = r -- B2 + + [W] F @Type alpha Char ~ Char + The `tcUnifyTysForInjectivity` will return substitution + [k :-> Type, r :-> Char, alpha :-> (g :: Type->Type) Int], with g unbound + If we used `substTy` on the LHS-tys for B2 we'd get + lhs_tys' = [Type, (g :: Type -> k) Int, Char] + That's ill-kinded because of the free `k`; and (worse) when `noEarlierMatch` matches + lhs_tys' against earlier equations, (B1) will fail to match. So (B2) will be + judged relevant, even though it isn't. Bad, bad. Use deep substitution. + +(CF9) Notice that, when generating the fundep equations for the solver to deal + with, we do /not/ apply the pre-unifying substitution S. The types wlhs,wrhs + contain meta-variables that we must actually unify! Example type family F a b where F [a] (Maybe b) = (a, G b) [W] F alpha beta ~ (Int, [gamma]) Pre-unification gives [alpha :-> [a], beta :-> Maybe b, a :-> Int] - This substitution eliminates the quantified variable `a` but not `b`, so we - want this: - FDEqns { fd_qtvs = [b], fd_eqs [ [Int] ~ alpha - , Maybe b ~ beta - , (Int,G b) ~ (Int, [gamma]) ] } - - Notice also that by applying the substitution to the template but not to the - Wanted, we uphold the `fd_eqs` invariant in `FDEqns` (see its data type decl) + We don't want to substitute `alpha` away! In fact we just generate these eqns: + FDEqns { fd_qtvs = [a,b], fd_eqs [ [a] ~ alpha + , Maybe b ~ beta + , (a, G b) ~ (Int, [gamma]) ] } + with `fd_qtvs` being the quantified variables of the equation, the LHSs + being the types from the equation and the RHSs being the types from the Wanted. + Notice that we thereby uphold the `fd_eqs` invariant in `FDEqns` (see its data + type decl). Note [Cache-caused loops] ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1517,8 +1586,8 @@ instantiateFunDepEqns (FDEqns { fd_qtvs = qtvs, fd_eqs = eqs }) rev_eqs = reverse eqs -- (reverse eqs): See Note [Reverse order of fundep equations] - -- The kinds of `qtvs` can have free variables, because of `trim_qtvs` - -- Example: T27828b + -- The kinds of `qtvs` can have free variables; + -- in the FDEqns produced by GHC.Tc.Instance.FunDeps.improveFromInstEnv empty_subst = mkEmptySubst $ mkInScopeSet $ tyCoVarsOfTypes (map tyVarKind qtvs) ===================================== testsuite/tests/typecheck/should_compile/T27994.hs ===================================== @@ -0,0 +1,48 @@ +{-# LANGUAGE AllowAmbiguousTypes #-} +{-# LANGUAGE DataKinds #-} +{-# LANGUAGE FunctionalDependencies #-} +{-# LANGUAGE GADTs #-} +{-# LANGUAGE TypeFamilies #-} +module T27994a (l2) where + +import Data.Kind (Constraint, Type) + +l1 :: [String] +l1 = ["hello", "world!"] + +l2 :: [Int] +l2 = gmap (HCons length HNil) l1 + +----- + +type HList :: [Type] -> Type +data HList p where + HNil :: HList '[] + HCons :: x -> HList xs -> HList (x:xs) + +type ($$) :: k -> [Type] -> Type +type family s $$ p where + (s :: Type) $$ '[] = s + (s :: Type -> k) $$ (a:b) = s a $$ b + +type Doms :: [Type] -> [Type] +type family Doms t where + Doms '[] = '[] + Doms ((a -> b) : t) = a : Doms t + +type Codoms :: [Type] -> [Type] +type family Codoms t where + Codoms '[] = '[] + Codoms ((a -> b) : t) = b : Codoms t + +type GMap :: k -> [Type] -> Constraint +class GMap (s :: k) t | t -> k where + gmap :: HList t -> s $$ Doms t -> s $$ Codoms t + +instance GMap [] '[a -> b] where + gmap :: HList '[a -> b] -> [a] -> [b] + gmap (HCons f HNil) = go + where + go :: [a] -> [b] + go [] = [] + go (x:xs) = f x : go xs ===================================== testsuite/tests/typecheck/should_compile/T27994a.hs ===================================== @@ -0,0 +1,53 @@ +{-# LANGUAGE DataKinds, TypeFamilies, PolyKinds, GADTs, AllowAmbiguousTypes #-} +module T27994 where + +import Data.Kind (Type) +import Data.Proxy (Proxy) + +type F :: forall{k}. k -> Type -> Type +type family F s x where + F (s :: Type -> k) (Proxy s) = Int + +f :: forall k (s :: k) x. F s x -> x +f _ = undefined + +g :: Proxy Maybe -> () +g _ = () + +data T a where + MkT :: T Int + +test :: T a -> () +test t = (\y -> case t of MkT -> g y) (f (3 :: Int)) + +{- [W] F @k_azX (s_azY::k_azX) p_azN ~ Int + +unify [*->k, s::*->k, Proxy @(*->k) (s::*->k), Int] + ~ [k_azX, s_azY::k_azX, p_azN, Int] + + --> [ k_azX :-> (* -> k) + , s :-> s_azY::k_azX + , p_azN :-> Proxy @(*->k) (s::*->k) ] + + fixpt --> [ k_azX :-> Type -> k + , s :-> s_azY::(Type -> k) + , p_azN :-> Proxy @(Type->k) (s::k_azX) ] + +Generate a [W] for each pair + [W] k ~ k_azX + [W] s ~ s_azY + [W] Proxy s ~ p_azN + [W] Int ~ Int +As an opt (claim) apply subst first + [W] k_azX ~ k_azX + [W] s_azY ~ s_azY + [W] Proxy s_azY ~ p_azN + [W] Int ~ Int + +-- [W] F @k_azX s_azY[tau] p_azN ~ Int +-- Want: fresh k' s' +-- [W] k' ~ k_azX +-- [W] s':k' ~ s_azY:k_azX +-- [W] Proxy s' ~ x +-} + ===================================== testsuite/tests/typecheck/should_compile/all.T ===================================== @@ -972,4 +972,6 @@ test('T27557', normal, compile, ['']) test('T27664', normal, compile, ['']) test('T27916', normal, compile, ['']) test('T26543_orig', normal, compile, ['']) +test('T27994', normal, compile, ['']) +test('T27994a', normal, compile, ['']) View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/565f804042073b1241adde95278e383… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/commit/565f804042073b1241adde95278e383… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0

[Git][ghc/ghc][wip/andreask/cfg-fix] 34 commits: Improve exprOkForSpeculation
by Sylvain Henry (＠hsyl20) 29 Oct '26

29 Oct '26

Sylvain Henry pushed to branch wip/andreask/cfg-fix at Glasgow Haskell Compiler / GHC Commits: 8c947573 by Simon Peyton Jones at 2026-10-16T13:11:53+01:00 Improve exprOkForSpeculation Fixes #27910, which exposed an infelicity in exprOkForSpeculation The new story is carefully described in new Note [exprOkForSpeculation], esp (a-c) and (EOKS1). - - - - - 7dedcf13 by Alan Zimmerman at 2026-10-16T19:03:17-04:00 EPA: Add TTG fields to FamilyInfo This commit is a simple add, with unused extension fields. It lays the groundwork for using them for exact print annotations - - - - - 26b9a8a0 by Arkissa at 2026-10-17T16:04:10+00:00 ghci: Prefer exact source spans in :type-at Prefer expression types for identical source spans. Fixes #27412. - - - - - 70dc4e61 by sheaf at 2026-10-18T18:15:15-04:00 Carefully implement coercion zapping This commit implements coercion zapping in the rewriter, controlled by the -fzap-coercions flag (on by default). This significantly reduces the size of coercions produced when rewriting type family applications, resulting in much faster compilation. See Note [Coercion zapping] in GHC.Tc.Solver.Rewrite for an explanation and further details. The rewriter now keeps track of the `CoVar`s and `CoercionHole`s it uses when rewriting, which allows computing the dependencies of a zapped coercion without traversing the coercion the rewriter produced, as explained in Note [Computing coercion dependencies] in GHC.Tc.Solver.Rewrite. Accompanying changes: - Rework how dependencies of 'UnivCo's are tracked, purely for performance reasons. We now use the 'UnivCoDeps' datatype instead of '[Coercion]', and we now insist that 'uco_deps' also contains the CoVars that appear in the LHS and RHS types of the UnivCo (as a caching mechanism). - Several performance improvements to the rewriter (e.g. increased strictness). - Improvements to the family application cache, including: - Better documentation. - Only cache a whole chain of reduction steps. - Don't put coercion holes in the cache, fixing #27518. - Allow Core Lint to be called from within the typechecker (and from within the coercion optimiser). - Move out "annotation linting" from Core Lint into the general Core pipeline, to avoid the dependency GHC.Core.Coercion.Opt->GHC.Core.Lint causing a regression in CountDepsParser due to transitive dependencies of GHC.Core.Opt.Monad and GHC.Unit.Module.ModGuts. Fixes #8095 #27336 #27518 Summary of performance numbers Compile-time byte allocated in GHC's testsuite (0 regressions) CoOpt_Singletons(normal) ghc/alloc 634,040,332 595,448,728 -6.1% GOOD FamAppCachePerf(normal) ghc/alloc 495,629,812 191,612,920 -61.3% GOOD LargeRecord(normal) ghc/alloc 1,226,806,434 966,260,976 -21.2% GOOD SimplCastPerf(normal) ghc/alloc 160,505,678 148,112,688 -7.7% GOOD T12227(normal) ghc/alloc 244,668,982 233,037,216 -4.8% GOOD T12545(normal) ghc/alloc 697,804,842 424,705,152 -39.1% GOOD T13386(normal) ghc/alloc 422,871,076 35,213,656 -91.7% GOOD T15703(normal) ghc/alloc 288,402,364 273,692,560 -5.1% GOOD T26426(normal) ghc/alloc 86,002,052 78,152,104 -9.1% GOOD T27336(normal) ghc/alloc 498,932,268 35,886,016 -92.8% GOOD T5030(normal) ghc/alloc 117,057,706 88,785,648 -24.2% GOOD T5321Fun(normal) ghc/alloc 252,552,628 243,820,176 -3.5% GOOD T8095(normal) ghc/alloc 768,198,564 127,779,520 -83.4% GOOD T9872a(normal) ghc/alloc 1,655,675,020 1,406,017,008 -15.1% GOOD T9872b(normal) ghc/alloc 1,864,180,628 1,644,706,696 -11.8% GOOD T9872b_defer(normal) ghc/alloc 2,834,585,468 1,657,729,760 -41.5% GOOD T9872c(normal) ghc/alloc 1,572,573,116 1,354,055,424 -13.9% GOOD T9872d(normal) ghc/alloc 351,516,004 331,071,424 -5.8% GOOD TcPlugin_RewritePerf(normal) ghc/alloc 2,063,065,680 1,847,464,272 -10.5% GOOD Chesskell (github.com/Proxymoron461/chesskell/issues) Typechecking that the first N plies of kholmovVsBronsteinShort are a valid chess game: Plies | GHC 9.14 time/alloc | This commit time/alloc | -----:|---------------------|------------------------| 2 | 41s/20GB | 0.2s/19MB | 4 | 95s/49GB | 0.3s/25MB | 6 | timed out | 0.6s/522MB | 8 | timed out | 7s/673MB | 10 | timed out | timed out | Compilation of singletons-base: compile time -28%, allocations -25% ------------------------- Metric Decrease: CoOpt_Singletons FamAppCachePerf LargeRecord SimplCastPerf T12227 T12545 T13386 T15703 T24984 T26426 T27336 T5030 T5321Fun T8095 T9872a T9872b T9872b_defer T9872c T9872d TcPlugin_RewritePerf ------------------------- Co-Authored-By: Sylvain Henry <sylvain(a)haskus.fr> - - - - - 3d326d7d by Sylvain Henry at 2026-10-18T18:16:09-04:00 JS: fix stack corruption in h$ap_gen_fast's pushAllRegs (#27624) The loop added in 9e46990976 ("JS: support more than 128 registers (#26558)") pushed register N to stack[sp - N - 2] while the switch it falls through into, and every reader of the h$ap_gen frame, uses stack[sp - (N - 2)]. Registers >= R32 were therefore written 4 slots too low: the slots the readers consult were left stale and the frame below was silently corrupted. Fix the loop to use the same slot formula as the switch, and add a regression test: an unknown call to a thunk with 35 argument register slots, which goes through the generic apply. Co-Authored-By: Claude Fable 5 <noreply(a)anthropic.com> - - - - - 6bbca569 by Andreas Klebinger at 2026-10-21T10:01:10+00:00 Fix a few tag inference bugs. * Treat bottoming functions and nullary join points as functions. * When generating bytecode don't rely on imported tag sigs. Fixes #27831, #27841 and #27842. - - - - - 71808e29 by Sylvain Henry at 2026-10-22T04:50:24-04:00 Add GHC.Internal.Unboxed.Do: do-notation for unboxed values (#27944) QualifiedDo operators can be representation-polymorphic: a do-block is expanded into ordinary applications before typechecking, and the only check on a bound variable is the deferred fixed-runtime-rep check. A representation-polymorphic (>>=) cannot be defined directly, though, because its body would bind a variable of unknown representation. The new module provides such operators for State#-passing code as methods of a class indexed by RuntimeRep, with one instance per representation. With optimisation the dictionaries disappear and a block compiles to the usual nested case expressions: Ubx.do { x <- readWordArray# mba 0# -- x :: Word# ; writeWordArray# mba 1# x -- unit actions are statements ; Ubx.pure x } Also add users guide documentation and tests. Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com> - - - - - 33247780 by Sylvain Henry at 2026-10-22T04:50:24-04:00 Bignum: use Ubx.do for State#-passing code (#27944) Rewrite the State#-passing code of the GHC.Internal.Bignum modules (Primitives, WordArray, BigNat, Natural, Integer, Backend.Native, Backend.GMP) with the qualified do-notation of GHC.Internal.Unboxed.Do. Nested case chains such as case newWordArray# sz s of { (# s, mwa #) -> case act mwa s of { s -> unsafeFreezeByteArray# mwa s }} become Ubx.do mwa <- newWordArray# sz act mwa unsafeFreezeByteArray# mwa The optimised Core is unchanged (compared per top-level binding). BigNat now enables MonoLocalBinds: local loops written with Ubx.do must stay monomorphic in the state type, otherwise GHC does not turn them into join points. Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com> - - - - - 593f5f5b by Sylvain Henry at 2026-10-22T04:51:16-04:00 JS: remove archive padding after wasm objects (#25665) On Darwin hosts, emar (llvm-ar) writes archives in the Darwin format when the first member is not a known object file, which is the case for the archives of the JS backend. This format pads each member to 8 bytes with '\n' and includes the padding in the member size. Wasm objects do not accept trailing bytes: wasm-ld failed with "section too large". The JS linker now removes this padding when it extracts a wasm object from an archive. In addition, Hadrian passes --format=gnu to emar for the JS target. See Note [Wasm objects in Darwin archives]. Also document the format of Cc (wasm) objects in Note [Cc object format]. Co-Authored-By: Claude Opus 5.5 <noreply(a)anthropic.com> - - - - - 13dff5d0 by L0neGamer at 2026-10-22T21:37:31-04:00 add options for resizing bytearrays (#27244) add five mutable bytearray resizing operations for various kinds of pinnedness and aligning. - prefer unpinned when allocating new array (``resizeMutableByteArrayPreferUnpinned#``; the ``resizeMutableByteArray#`` default) - prefer pinned when allocating new array (``resizeMutableByteArrayPreferPinned#``) - prefer aligned and pinned when allocating new array (``resizeMutableByteArrayPreferAlignedPinned#``) - prefer same as input bytearray when allocating new array (``resizeMutableByteArrayKeepPinnedness#``) - ensure the output array is aligned and pinned after operation (``resizeMutableByteArrayMakeAlignedPinned#``) - - - - - 0c17201c by Sylvain Henry at 2026-10-22T21:38:23-04:00 JS: implement performMinorGC and performBlockingMajorGC (#28012) GHC.Internal.System.Mem imports three C entry points: performMajorGC, performGC (for performMinorGC) and performBlockingMajorGC. The JS RTS only defined h$performMajorGC, so calling performMinorGC or performBlockingMajorGC threw "ReferenceError: h$performGC is not defined" at runtime. The JS RTS has a single non-generational collector, so both missing entry points are now aliases of h$performMajorGC. Add a test that calls every GC entry point of System.Mem. Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com> - - - - - 7a631ff2 by L0neGamer at 2026-10-23T09:11:39-04:00 Mon(a|oi)d of No (Return|Mappend) progress (!15795) Make non-canonical definitions of return and mappend non-suppressible errors, and make canonical definitions of those methods warnings. CLC proposal: https://github.com/haskell/core-libraries-committee/issues/328 - - - - - 14656fd4 by Simon Jakobi at 2026-10-23T09:12:27-04:00 Cmm: keep the CBE substitution idempotent When blocks are merged over multiple rounds during common block elimination, a block that won a merge in an earlier round can still lose a merge in a later round. The substitution map (Subst) used to record this as a chain: the earlier loser mapped to a block that was itself eliminated. In PLT terminology, the substitution became *non-idempotent*. Applying such a substitution with a single lookup left edges to unreachable blocks in the Cmm graph, eventually leading to a setInfoTableStackMap panic (#27368). With this patch, each round's fresh substitutions are applied to the old ones as they are added (extendSubst), so every eliminated block maps directly to its final replacement and the substitution stays idempotent. Fixes #27368. Assisted-by: Claude Fable 5.1 - - - - - 92848e61 by Evgeny Malyshev at 2026-10-23T20:26:04-04:00 hadrian: Include transitive dependencies in Haddock builds Haddock needs indirect dependency interfaces to resolve references to definitions outside a package's direct dependencies. Expose Cabal's configured dependency closure through ContextData and use it alongside the direct list, preserving direct-interface precedence. Bump Shake's cache version for the serialized oracle change. Add a Hadrian selftest requiring template-haskell's indirect ghc-prim interface. Fixes #25850. Fixes #27786. Assisted-By: OpenAI Codex Metric Increase: haddock.Cabal - - - - - 5e8ae81b by Zubin Duggal at 2026-10-23T20:26:47-04:00 WinIO: Don't confuse Win32 errors and NTSTATUS codes WinIO misinterpreted NTSTATUS codes as Win32 errors, and had some ad hoc logic to deal with this. However, this logic was incomplete and resulted in the mishandling of results of certain I/O operations (see testcase). Now the `ErrCode` argument in `CompletionCallback` is always the Win32 error code, and we convert the NTSTATUS using RtlNtStatusToDosError. Also when an overlapped/async operation such as ReadFile fails immediately, with a Win32 error such as ERROR_BROKEN_PIPE, we don't get a further completion notification, so we never recorded the failure. This worked by coincidence earlier, because `STATUS_PENDING` happened to have the same numerical code as `ERROR_NO_MORE_ITEMS`, but it doesn't now that we translate `STATUS_PENDING` into the correct Win32 error code (`ERROR_IO_PENDING`). Fix this by using the Win32 error value we obtained when an operation returns `STATUS_PENDING`. Fixes #27971 - - - - - c42ed66a by Martijn Bastiaan at 2026-10-24T16:00:42+02:00 Add single-threaded test for #27526 + #27123 Issue #27526 is a duplicate of #27123, which has already been fixed in HEAD. The proposed patch in the former carries a "nicer" regression test in that it doesn't rely on multi-threading to trigger. Closes #27526 - - - - - 7f73f515 by Simon Peyton Jones at 2026-10-24T17:29:19-04:00 Refactor QuickLook a bit This MR simply refactors QuickLook a bit. In particular, it simplifies `tcApp`, and moves shared code into `finishApp`. It also adds `Note [QuickLook: arguments before result]` responding to #26599. There should be no user-visible changes, except perhaps in error messages, which improve marginally. For some reason T24984 compiles 8% faster. I'll take the win! Metric Decrease: T24984 - - - - - a4a3a964 by Alan Zimmerman at 2026-10-24T17:30:03-04:00 EPA: Use AnnList for XClosedTypeFamily We now have TTG annotations for FamilyInfo. Make use of it to store the AnnList annotations for a closed type family. - - - - - 953e4c7f by sheaf at 2026-10-26T18:19:10-04:00 Rework LogQueue to be a proper queue Now 'LogQueue' uses a 'TQueue' with a separate 'TVar' to indicate whether the log queue is finished logging, instead of the previous weird data representation using a reverse-order '[Maybe Msg]' with 'Nothing' denoting the end of the queue. - - - - - ec29c5fa by Rodrigo Mesquita at 2026-10-26T18:19:10-04:00 Refactor GHC.Driver.MakeAction Pull out of 'runParPipelines' the logic for creating a 'MakeEnv' ready to be used by multiple threads, as that will be useful for downsweep as well (which doesn't fit the 'runPipelines' flow), rather than being just for upsweep. The code is moved and re-structured to match the export list, simplify the sequentiality checks previously both in 'runPipelines' and 'runAllPipelines', which were weirdly similar and confusing; into the two part step where we need these checks: (1) to construct the MakeEnv, (2) to run the MakeActions in parallel. These two steps are separate and used to be too mixed up. Some additional little simplifications or clean ups here and there. - - - - - 3fb22073 by Rodrigo Mesquita at 2026-10-26T18:19:11-04:00 Refactor GHC.Driver.Downsweep in preparation for parallel downsweep Pure refactor to improve the code to facilitate implementing parallel downsweep in the next commit. This commit puts a MakeEnv into the DownsweepEnv, gives the fields proper names and uses RecordWildcards to simplify, rather than passing around all diagnostic wrappers, driver-message-things and using 10s of positional fields. No behavior changes here! - - - - - 52f385a7 by sheaf at 2026-10-26T18:19:11-04:00 Driver: enableCodeGenForTH in load', not downsweep enableCodeGenForTH modifies the module graph for planning, creating fresh temporary paths. So it should only be run when we are about to execute a build plan (e.g. in load'), so that consumers that want to know facts about the module graph don't go looking for files that don't exist yet (in particular 'haddock --no-compilation'). - - - - - 82402d14 by sheaf at 2026-10-26T18:19:11-04:00 2-phase Cache/Search Finder monad This commit restructures the finder abstraction by introducing the 'FinderM' monad, which splits module lookup operations into two phases: - a cache-only phase, performing no filesystem access, - from the first cache miss onwards, a search action which may access the filesystem. This allows consumers to distinguish between quick cached results versus more expensive filesystem search operations. - - - - - 5c302b40 by sheaf at 2026-10-26T18:19:11-04:00 Separate Home/External finder caches The caches for home modules and modules from external units have different lifetimes, needing to be invalidated at different points. This commit moves the cache for modules from external units into the 'UnitState' type, which keeps the cache correct whenever package flags or package databases change (setSessionDynFlags, :set -package ..., Backpack's addUnit). This also makes it easier to clear the cache of home modules, as they are now stored separately (obviating the 'isUnitEnvInstalledModule' check we used to perform). - - - - - 605fbb96 by sheaf at 2026-10-26T18:19:11-04:00 Add known home modules to the finder cache The finder looks in the file system for the on-disk source that corresponds to a Haskell module name. There are two kinds of modules where we need to bypass this search: - a file passed directly in the command line, which may not be in any search path and whose module name may differ from the file name; - Backpack signatures and modules, which do not exist as files at all. This commit handles these two situations by adding an immutable 'KnownHomeModules' in the finder cache, which is always consulted first before doing any search. This removes the ad-hoc imperative logic which relied on mutating the finder cache at careful points. 'setModuleGraph' now also takes the known home modules, so that the module graph and the known home modules are always set consistently. See Note [Known home modules] in GHC.Unit.Finder.Types. - - - - - 82718f44 by sheaf at 2026-10-26T18:19:11-04:00 Make 'UnresolvedImport' into a suitable cache key The goal of this commit is to make 'UnresolvedImport' suitable for use as a cache key. Specifically, during downsweep we want to be able to cache module resolution queries. We want to do this using some kind of map, e.g. (roughly speaking): Map UnresolvedImport ResolvedImport For this to work, 'UnresolvedImport' must contain **precisely** the input data to each resolution computation. In other words: 1. If something changes that materially affects resolution, it must be stored in the cache key. 2. If changing some input does not materially affect the outcome, then it must not be stored in the cache key. To satisfy (1), we change 'UnitNode' to also keep track of the home unit in whose unit state the unit's dependencies were looked up. For (2), 'ImportResolution' now stores a 'ModuleLookupScope' instead of an 'UnresolvedImportOrigin'. Plugin imports are expressed with a new 'LookupPlugin' scope. The source location of an import, which is immaterial to the resolution, is stored out of band outside 'UnresolvedImport', in 'ModSummary'. - - - - - d4635c7c by sheaf at 2026-10-26T18:19:11-04:00 Add GHC.Data.Dependent This commit adds a minimal implementation of dependent sums ('Some', 'DSum') and dependent maps ('DMap'). The implementation is not entirely performance optimal, as there are a few extraneous allocations compared to the full-blown implementation in the 'dependent-map' package. For our use case (caching downsweep computations), this does not matter much. See GHC.Data.Dependent. - - - - - 6759533e by sheaf at 2026-10-26T18:19:11-04:00 Driver: structured concurrent worker abstraction This commit introduces a structured concurrency framework in the style of the 'ki' library: a collection of threads within a scope. We implement two kinds of concurrent workers on top of this framework: - Independent workers cannot wait for one another at all. The only scheduling operation is to wait for quiescence. - Coordinating workers declare an STM readiness condition (waiting on other workers to complete) which gates their start. See Note [Deterministic concurrent workers] in GHC.Driver.Concurrency. This commit ports upsweep to this new framework, with downsweep being left as subsequent work. Further changes along the way: - Refactoring of how concurrency is acquired to avoid the footgun of trying to use a no-op 'AbstractSem' as a lock in the serial case. - The "re-run with -j1" logic for semaphore opening errors no longer triggers on late semaphore failures (part-way through a lengthy computation). - Logger threads are properly cleaned up on exception, with each concurrent worker's log queue and local TmpFs properly bracketed. - The 'GhcMessage -> AnyGhcDiagnostic' and 'Maybe Messager' arguments of 'depanalE', 'depanalPartial' and 'downsweep', which were all dead in practice, have been dropped. - Concurrent logging has been completely overhauled. Each worker has its own log queue. Workers whose logs are printed as we go are spawned in the order in which their logs must be printed, now using a TQueue instead of the old complicated LogQueueQueue mechanism. 'logThread' is replaced by 'withLogPrinter', which runs the thread that prints the log queues within a concurrent worker scope. These changes allow us to fix issues with the exception handling: we now always flush remaining log queues before exiting (even when the action throws an exception), and exceptions that arise while printing are properly rethrown. This fixes #28009. - - - - - 38a99006 by sheaf at 2026-10-26T18:19:11-04:00 Rule-based deterministic concurrent downsweep This commit rewrites downsweep as a single query-answering rule (see 'DownsweepRule') that can be executed by concurrent worker threads. The design allows every expensive operation (preprocessing files with CPP, parsing headers, reading interfaces) to be performed concurrently according to the -j<N>/-jsem flags. See Note [Rules-based downsweep] in GHC.Driver.Downsweep. The rules are run by "GHC.Driver.Concurrency.runRules" in a demand-driven way: each worker can demand other workers to run, but never wait on another worker. Every piece of work is done at most once. Fixes #27514 Memory usage can increase due to doing more work concurrently: ------------------------- Metric Increase: MultiComponentModulesRecomp MultiComponentModulesRecomp100 mhu-perf ------------------------- - - - - - 9fa7b7be by sheaf at 2026-10-26T18:19:11-04:00 Acquire/release semaphore tokens more readily This commit reworks GHC.Driver.MakeSem to use the scoped worker abstraction of GHC.Utils.Concurrent.Scope. This allows us to get rid of a lot of the tricky logic in GHC.Driver.MakeSem involving manual exception handling, thread spawning and lifetime management, etc. The main user-visible change is that token acquisition is no longer rate limited (due to a bug, there previously was a 1s rate limit on acquisition of semaphore tokens), with the release debounce period shortened from 1s to 10ms. Other changes: - With -jsem, the number of capabilities is now capped at the number of processors, bringing it in line with the -jN behaviour. - Pending jobs whose wait for a token is interrupted by an exception are now withdrawn from the jobserver, instead of acquiring a token that it hung onto without doing useful work. Fixes #27763 - - - - - c83c6733 by Alan Zimmerman at 2026-10-26T18:19:55-04:00 EPA: Update AnnClsInstDecl to contain AnnList It was already separately tracking the '{', '}' and leading ';' values. This aligns it with layout-introducing parser productions ('vocurly'), so the layout processing can be handled uniformly - - - - - ac870ad5 by Simon Jakobi at 2026-10-28T04:40:42-04:00 X86 NCG: use 32-bit POPCNT for sub-word popcounts A 32-bit write zero-extends the result to the full register, so we don't need to emit additional code for this. We also skip emitting zero-extension instructions for the argument when it is already visibly zero-extended. For the byte-wise popcount loop from #24813, this shrinks the popcount sequence from 5 instructions to 3. Assisted-by: Claude Fable 5.1 - - - - - 53087373 by Facundo Domínguez at 2026-10-28T04:41:32-04:00 Use a ShortByteString in Serialized instead of [Word8] This makes the in-memory representation 40x more compact for all values stored in annotations. A test is added to measure the representation size of annotations specified in the compiler input. The GHC API is modified. All functions that previously handled annotations in [Word8] form now use ShortByteString. The following definitions have modified types: * GHC.findGlobalAnns * GHC.Tc.Gen.Splice.getAnnotationsByTypeRep * GHC.Core.Opt.Monad.getAnnotations * GHC.Core.Opt.Monad.getFirstAnnotations * GHC.Types.Annotations.deserializeAnns * GHC.Types.Annotations.findAnns * GHC.Types.Annotations.findAnnsByTypeRep * GHC.Serialized.Serialized * GHC.Serialized.toSerialized * GHC.Serialized.fromSerialized * GHC.Serialized.serializeWithData * GHC.Serialized.deserializeWithData * GHCi.Message.ReifyAnnotations Fixes #28002. - - - - - 9803f87b by Andreas Klebinger at 2026-10-29T09:42:30+01:00 x86 ncg: Track the current block in NatM to keep the CFG in sync The x86 NCG maintains a CFG that is used for code layout. Whenever instruction selection introduces new basic blocks (for instance for word-to-float conversions, cmpxchg based atomic operations, ctz on i386 or 64-bit shifts on i386) it must update this CFG, and the code that follows must use the new block as its origin. Before this patch the current block was threaded explicitly through some statement-level functions only. Block-introducing primops that were dispatched as "simple" (word2Float#, #27915) and the 64-bit shifts in iselExpr64 did not update the CFG at all. This resulted in an out-of-sync CFG, bad code layout and the "Can't find weight for edge that should have one" panic once the register allocator inserted fixup blocks (#27694, #27915). We now keep the current block in the NatM state and provide a few combinators (continueInNewBlock, addCondBlock, addDiamondFlow, addColdSelfLoop, increaseEdgeWeight) that update the CFG and the current block at the same time. See Note [Updating the CFG during CodeGen] in GHC.CmmToAsm.Monad. The cfg sanity check (-dasm-lint) now also checks that all branches in the generated code have a matching edge in the CFG, which catches this class of bug early. It immediately found another desync: when the register allocator needs more stack than preallocated, allocMoreStack moves the body of every entry block into a new block and retargets the local jumps to it, but the CFG only recorded the new block as the immediate successor of the entry block. The new addPreentryBlock also redirects the jump edges (and keeps the call edges, since returns still go through the entry block). Fixes #27694, #27915 Co-authored-by: Sylvain Henry <sylvain(a)haskus.fr> Co-Authored-By: Claude Fable 5.1 <noreply(a)anthropic.com> - - - - - 377 changed files: - + changelog.d/25665 - + changelog.d/25850 - + changelog.d/27368 - + changelog.d/27624 - + changelog.d/27694 - + changelog.d/27971 - + changelog.d/28012 - + changelog.d/T27412 - + changelog.d/T27763 - + changelog.d/T27831-bottoming-strict-field - + changelog.d/T27842-bytecode-imported-tags - + changelog.d/T28002 - + changelog.d/T28002-api - + changelog.d/monad-of-no-return-canonical - + changelog.d/ncg-x86-subword-popcnt - + changelog.d/parallel-downsweep - + changelog.d/resizeMutableByteArray-pinnedness - + changelog.d/unboxed-do - + changelog.d/zap-coercions - compiler/GHC.hs - compiler/GHC/Builtin.hs - compiler/GHC/Builtin/primops.txt.pp - compiler/GHC/ByteCode/Asm.hs - compiler/GHC/Cmm/CommonBlockElim.hs - compiler/GHC/CmmToAsm.hs - compiler/GHC/CmmToAsm/AArch64/CodeGen.hs - compiler/GHC/CmmToAsm/BlockLayout.hs - compiler/GHC/CmmToAsm/CFG.hs - compiler/GHC/CmmToAsm/CFG/Dominators.hs - compiler/GHC/CmmToAsm/Monad.hs - compiler/GHC/CmmToAsm/X86/CodeGen.hs - compiler/GHC/CmmToAsm/X86/Instr.hs - compiler/GHC/CmmToLlvm/CodeGen.hs - compiler/GHC/Core.hs - compiler/GHC/Core/Coercion.hs - compiler/GHC/Core/Coercion.hs-boot - compiler/GHC/Core/Coercion/Opt.hs - compiler/GHC/Core/FamInstEnv.hs - compiler/GHC/Core/Lint.hs - + compiler/GHC/Core/Lint.hs-boot - compiler/GHC/Core/Opt/Monad.hs - compiler/GHC/Core/Opt/Pipeline.hs - compiler/GHC/Core/Reduction.hs - compiler/GHC/Core/SimpleOpt.hs - compiler/GHC/Core/TyCo/FVs.hs - compiler/GHC/Core/TyCo/FVs.hs-boot - compiler/GHC/Core/TyCo/Make.hs - compiler/GHC/Core/TyCo/Rep.hs - compiler/GHC/Core/TyCo/Rep.hs-boot - compiler/GHC/Core/TyCo/Subst.hs - compiler/GHC/Core/TyCo/Tidy.hs - compiler/GHC/Core/Type.hs - compiler/GHC/Core/Utils.hs - compiler/GHC/CoreToIface.hs - + compiler/GHC/Data/Dependent.hs - compiler/GHC/Data/FastString.hs - compiler/GHC/Data/OrdList.hs - compiler/GHC/Data/Pair.hs - compiler/GHC/Data/Word64Map/Internal.hs - compiler/GHC/Data/Word64Set/Internal.hs - compiler/GHC/Driver/Backpack.hs - compiler/GHC/Driver/CmdLine.hs - + compiler/GHC/Driver/Concurrency.hs - compiler/GHC/Driver/Config.hs - + compiler/GHC/Driver/Config/Concurrency.hs - compiler/GHC/Driver/Config/Core/Lint.hs - compiler/GHC/Driver/Downsweep.hs - compiler/GHC/Driver/DynFlags.hs - compiler/GHC/Driver/Env.hs - compiler/GHC/Driver/Errors/Ppr.hs - compiler/GHC/Driver/Errors/Types.hs - compiler/GHC/Driver/Flags.hs - compiler/GHC/Driver/Main/Passes.hs - compiler/GHC/Driver/Make.hs - compiler/GHC/Driver/MakeAction.hs - compiler/GHC/Driver/MakeFile.hs - compiler/GHC/Driver/MakeSem.hs - compiler/GHC/Driver/Pipeline/Execute.hs - compiler/GHC/Driver/Pipeline/LogQueue.hs - compiler/GHC/Driver/Session.hs - compiler/GHC/Hs/Decls.hs - compiler/GHC/Hs/Dump.hs - compiler/GHC/Hs/Utils.hs - compiler/GHC/HsToCore/Pmc/Types.hs - compiler/GHC/HsToCore/Quote.hs - compiler/GHC/Iface/Ext/Ast.hs - compiler/GHC/Iface/Load.hs - compiler/GHC/Iface/Recomp.hs - compiler/GHC/Iface/Rename.hs - compiler/GHC/Iface/Syntax.hs - compiler/GHC/Iface/Type.hs - compiler/GHC/IfaceToCore.hs - compiler/GHC/JS/Opt/Simple.hs - compiler/GHC/Linker/Deps.hs - compiler/GHC/Parser.y - compiler/GHC/Parser/Header.hs - compiler/GHC/Parser/PostProcess.hs - compiler/GHC/Rename/Env.hs - compiler/GHC/Rename/Module.hs - compiler/GHC/Runtime/Eval/Types.hs - compiler/GHC/Runtime/Interpreter/JS.hs - compiler/GHC/Runtime/Loader.hs - compiler/GHC/Stg/EnforceEpt.hs - compiler/GHC/Stg/EnforceEpt/Rewrite.hs - compiler/GHC/Stg/EnforceEpt/TagSig.hs - compiler/GHC/Stg/EnforceEpt/Types.hs - compiler/GHC/StgToCmm/Prim.hs - compiler/GHC/StgToJS/Apply.hs - compiler/GHC/StgToJS/Linker/Linker.hs - compiler/GHC/StgToJS/Object.hs - compiler/GHC/StgToJS/Prim.hs - compiler/GHC/Tc/Deriv/Generate.hs - compiler/GHC/Tc/Errors/Ppr.hs - compiler/GHC/Tc/Errors/Types.hs - compiler/GHC/Tc/Gen/App.hs - compiler/GHC/Tc/Gen/Splice.hs - compiler/GHC/Tc/Instance/Family.hs - compiler/GHC/Tc/Module.hs - compiler/GHC/Tc/Plugin.hs - compiler/GHC/Tc/Solver/Default.hs - compiler/GHC/Tc/Solver/Equality.hs - compiler/GHC/Tc/Solver/InertSet.hs - compiler/GHC/Tc/Solver/Monad.hs - compiler/GHC/Tc/Solver/Rewrite.hs - compiler/GHC/Tc/Solver/Solve.hs - compiler/GHC/Tc/TyCl.hs - compiler/GHC/Tc/Types.hs - compiler/GHC/Tc/Types/Constraint.hs - compiler/GHC/Tc/Utils/Backpack.hs - compiler/GHC/Tc/Utils/TcMType.hs - compiler/GHC/ThToHs.hs - compiler/GHC/Types/Annotations.hs - compiler/GHC/Types/Basic.hs - compiler/GHC/Types/Error/Codes.hs - compiler/GHC/Types/Hint.hs - compiler/GHC/Types/Hint/Ppr.hs - compiler/GHC/Types/Unique/DSet.hs - compiler/GHC/Types/Unique/FM.hs - compiler/GHC/Types/Unique/Map.hs - compiler/GHC/Types/UnresolvedImport.hs - compiler/GHC/Types/Var/Set.hs - compiler/GHC/Unit/External/ModuleOrigin.hs - compiler/GHC/Unit/External/Visibility.hs - compiler/GHC/Unit/Finder.hs - + compiler/GHC/Unit/Finder/Cache.hs - compiler/GHC/Unit/Finder/Types.hs - compiler/GHC/Unit/Module/Graph.hs - compiler/GHC/Unit/Module/ModSummary.hs - compiler/GHC/Unit/Module/WholeCoreBindings.hs - compiler/GHC/Unit/State.hs - compiler/GHC/Unit/Types.hs - + compiler/GHC/Utils/Concurrent/Scope.hs - compiler/GHC/Utils/Monad/Codensity.hs - compiler/GHC/Utils/Outputable.hs - compiler/GHC/Utils/Ppr/Colour.hs - compiler/GHC/Utils/TmpFs.hs - compiler/Language/Haskell/Syntax/Decls.hs - compiler/Language/Haskell/Syntax/Extension.hs - compiler/ghc.cabal.in - docs/users_guide/extending_ghc.rst - docs/users_guide/exts/qualified_do.rst - docs/users_guide/exts/type_families.rst - docs/users_guide/using-warnings.rst - ghc/GHCi/UI.hs - ghc/GHCi/UI/Info.hs - ghc/Main.hs - ghc/ghc-bin.cabal.in - hadrian/src/Hadrian/Haskell/Cabal/Parse.hs - hadrian/src/Hadrian/Haskell/Cabal/Type.hs - hadrian/src/Main.hs - hadrian/src/Rules/Documentation.hs - hadrian/src/Rules/Selftest.hs - hadrian/src/Settings/Builders/Ar.hs - hadrian/src/Settings/Warnings.hs - libraries/base/src/GHC/Base.hs - libraries/base/src/GHC/Exts.hs - + libraries/base/tests/IO/T27971.hs - + libraries/base/tests/IO/T27971.stdout - libraries/base/tests/IO/all.T - libraries/ghc-boot/GHC/Serialized.hs - libraries/ghc-internal/CHANGELOG.md - libraries/ghc-internal/ghc-internal.cabal.in - libraries/ghc-internal/src/GHC/Internal/Base.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Backend/GMP.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Backend/Native.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/BigNat.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Integer.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Natural.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/Primitives.hs - libraries/ghc-internal/src/GHC/Internal/Bignum/WordArray.hs - libraries/ghc-internal/src/GHC/Internal/Event/Windows.hsc - libraries/ghc-internal/src/GHC/Internal/Event/Windows/FFI.hsc - libraries/ghc-internal/src/GHC/Internal/IO/Windows/Handle.hsc - + libraries/ghc-internal/src/GHC/Internal/Unboxed/Do.hs - libraries/ghci/GHCi/Message.hs - libraries/ghci/GHCi/TH.hs - libraries/ghci/GHCi/TH/Binary.hs - linters/lint-codes/LintCodes/Static.hs - rts/PrimOps.cmm - rts/RtsSymbols.c - rts/include/stg/MiscClosures.h - rts/js/environment.js - testsuite/tests/ado/ado-optimal.hs - testsuite/tests/ado/ado001.hs - testsuite/tests/annotations/should_run/Makefile - + testsuite/tests/annotations/should_run/T16841.hs - + testsuite/tests/annotations/should_run/T16841.stdout - testsuite/tests/annotations/should_run/all.T - + testsuite/tests/bytecode/T27842.hs - + testsuite/tests/bytecode/T27842.stdout - + testsuite/tests/bytecode/T27842A.hs - + testsuite/tests/bytecode/T27842N.hs - testsuite/tests/bytecode/all.T - + testsuite/tests/codeGen/should_compile/T27368.hs - + testsuite/tests/codeGen/should_compile/T27694.hs - testsuite/tests/codeGen/should_compile/all.T - + testsuite/tests/codeGen/should_gen_asm/T24813.asm - + testsuite/tests/codeGen/should_gen_asm/T24813.cmm - testsuite/tests/codeGen/should_gen_asm/all.T - + testsuite/tests/codeGen/should_run/T27624.hs - + testsuite/tests/codeGen/should_run/T27624.stdout - + testsuite/tests/codeGen/should_run/T27915.hs - + testsuite/tests/codeGen/should_run/T27915.stdout-ws-32 - + testsuite/tests/codeGen/should_run/T27915.stdout-ws-64 - testsuite/tests/codeGen/should_run/all.T - testsuite/tests/concurrent/T13615/Parallel.hs - testsuite/tests/concurrent/should_run/T21651.hs - testsuite/tests/concurrent/should_run/setnumcapabilities001.hs - testsuite/tests/corelint/T27374.hs - testsuite/tests/count-deps/CountDepsParser.stdout - testsuite/tests/cpranal/should_run/T20598a.hs - testsuite/tests/dependent/should_compile/T14729.stderr - testsuite/tests/deriving/should_compile/T13758.hs - testsuite/tests/deriving/should_run/all.T - + testsuite/tests/deriving/should_run/drvrun-canonical-methods.hs - + testsuite/tests/deriving/should_run/drvrun-canonical-methods.stderr - + testsuite/tests/deriving/should_run/drvrun-canonical-methods.stdout - testsuite/tests/determinism/determ017/A.hs - testsuite/tests/diagnostic-codes/codes.stdout - testsuite/tests/dmdanal/should_compile/T9208.hs - testsuite/tests/driver/Makefile - testsuite/tests/driver/T27461/Makefile - + testsuite/tests/driver/T27461/T27461c.stderr - testsuite/tests/driver/T27461/all.T - + testsuite/tests/driver/T27461/src/Wrong.hs - + testsuite/tests/driver/T28009.hs - testsuite/tests/driver/all.T - testsuite/tests/generics/T27245.hs - testsuite/tests/generics/T27245.stdout - testsuite/tests/ghc-api/downsweep/IncrementalDownsweep.hs - testsuite/tests/ghc-api/downsweep/OldModLocation.hs - testsuite/tests/ghc-api/downsweep/PartialDownsweep.hs - testsuite/tests/ghc-api/fixed-nodes/FixedNodes.hs - testsuite/tests/ghc-api/fixed-nodes/InterfaceModuleGraph.hs - testsuite/tests/ghc-api/fixed-nodes/ModuleGraphInvariants.hs - testsuite/tests/ghci.debugger/scripts/T14628.stderr - testsuite/tests/ghci/prog-mhu001/prog-mhu001c.stdout - testsuite/tests/ghci/prog-mhu002/prog-mhu002d.stdout - testsuite/tests/ghci/scripts/T16575.script - testsuite/tests/ghci/scripts/T16575.stdout - testsuite/tests/ghci/scripts/T16804.stdout - testsuite/tests/ghci/scripts/T20150.stdout - + testsuite/tests/ghci/scripts/T27412.hs - + testsuite/tests/ghci/scripts/T27412.script - + testsuite/tests/ghci/scripts/T27412.stdout - testsuite/tests/ghci/scripts/all.T - testsuite/tests/haddock/perf/Fold.hs - testsuite/tests/haddock/should_compile_flag_haddock/T17544.stderr - testsuite/tests/hiefile/should_compile/hie002.hs - testsuite/tests/hpc/raytrace/Eval.hs - testsuite/tests/indexed-types/should_compile/T3787.hs - + testsuite/tests/indexed-types/should_compile/ZapDeps.hs - testsuite/tests/indexed-types/should_compile/all.T - testsuite/tests/indexed-types/should_fail/T7729.hs - testsuite/tests/indexed-types/should_fail/T7729a.hs - testsuite/tests/interface-stability/ghc-experimental-exports.stdout - testsuite/tests/interface-stability/ghc-experimental-exports.stdout-mingw32 - testsuite/tests/interface-stability/ghc-prim-exports.stdout - testsuite/tests/interface-stability/ghc-prim-exports.stdout-mingw32 - testsuite/tests/javascript/Makefile - + testsuite/tests/javascript/T25665.hs - + testsuite/tests/javascript/T25665.pkg - + testsuite/tests/javascript/T25665.stdout - + testsuite/tests/javascript/T25665Lib.hs - + testsuite/tests/javascript/T25665_c.c - + testsuite/tests/javascript/T25665_js.js - testsuite/tests/javascript/all.T - testsuite/tests/module/mod133.hs - testsuite/tests/parser/should_compile/DumpParsedAst.stderr - testsuite/tests/parser/should_compile/DumpRenamedAst.stderr - testsuite/tests/parser/should_compile/KindSigs.stderr - testsuite/tests/perf/compiler/T13253.hs - testsuite/tests/perf/compiler/T14683.hs - testsuite/tests/perf/compiler/T16473.hs - testsuite/tests/perf/compiler/T18730_A.hs - testsuite/tests/perf/compiler/T25723.hs - testsuite/tests/plugins/T26761Plugin.hs - testsuite/tests/plugins/all.T - testsuite/tests/plugins/defaulting-plugin/DefaultLifted.hs - testsuite/tests/pmcheck/should_compile/T11276.hs - + testsuite/tests/primops/should_run/ResizeByteArrayPinnedness.hs - + testsuite/tests/primops/should_run/ResizeByteArrayPinnedness.stdout - testsuite/tests/primops/should_run/all.T - testsuite/tests/printer/Test24533.stdout - testsuite/tests/profiling/should_compile/T19894/StreamK.hs - testsuite/tests/profiling/should_run/T3001-2.hs - testsuite/tests/programs/galois_raytrace/Eval.hs - testsuite/tests/qualifieddo/should_compile/all.T - + testsuite/tests/qualifieddo/should_compile/qdocompile003.hs - + testsuite/tests/qualifieddo/should_compile/qdocompile003.stderr - testsuite/tests/qualifieddo/should_fail/all.T - + testsuite/tests/qualifieddo/should_fail/qdofail006.hs - + testsuite/tests/qualifieddo/should_fail/qdofail006.stderr - testsuite/tests/qualifieddo/should_run/all.T - + testsuite/tests/qualifieddo/should_run/qdorun008.hs - + testsuite/tests/qualifieddo/should_run/qdorun008.stdout - testsuite/tests/rebindable/rebindable2.hs - + testsuite/tests/rename/should_fail/Noncanonical.hs - + testsuite/tests/rename/should_fail/Noncanonical.stderr - testsuite/tests/rename/should_fail/all.T - testsuite/tests/rename/should_fail/rnfail026.hs - testsuite/tests/rep-poly/RepPolyRule3.stderr - + testsuite/tests/rts/T27526.hs - + testsuite/tests/rts/T27526.stdout - + testsuite/tests/rts/T28012.hs - + testsuite/tests/rts/T28012.stdout - testsuite/tests/rts/all.T - testsuite/tests/rts/ipe/IpeStats/Fold.hs - testsuite/tests/simplCore/should_compile/T14908_Deps.hs - testsuite/tests/simplCore/should_compile/T26051.stderr - + testsuite/tests/simplCore/should_compile/T27910.hs - + testsuite/tests/simplCore/should_compile/T27910.stderr - testsuite/tests/simplCore/should_compile/all.T - testsuite/tests/simplCore/should_compile/simpl017.hs - testsuite/tests/simplStg/should_compile/all.T - + testsuite/tests/simplStg/should_compile/inferTags005.hs - + testsuite/tests/simplStg/should_compile/inferTags005.stderr - + testsuite/tests/simplStg/should_compile/inferTags006.hs - + testsuite/tests/simplStg/should_compile/inferTags006.stderr - + testsuite/tests/simplStg/should_run/InferTags007_A.hs - + testsuite/tests/simplStg/should_run/T27831.hs - + testsuite/tests/simplStg/should_run/T27831.stdout - + testsuite/tests/simplStg/should_run/T27857.hs - + testsuite/tests/simplStg/should_run/T27857.stdout - testsuite/tests/simplStg/should_run/all.T - + testsuite/tests/simplStg/should_run/inferTags007.hs - + testsuite/tests/simplStg/should_run/inferTags007.stdout - testsuite/tests/splice-imports/SI35.hs - testsuite/tests/typecheck/should_compile/DeepSubsumption02.hs - testsuite/tests/typecheck/should_compile/T4524.hs - testsuite/tests/typecheck/should_compile/T4969.hs - testsuite/tests/typecheck/should_fail/T22707.hs - testsuite/tests/typecheck/should_fail/T22707.stderr - testsuite/tests/typecheck/should_fail/T25950.stderr - testsuite/tests/typecheck/should_fail/T7019.hs - testsuite/tests/typecheck/should_fail/T8450.stderr - testsuite/tests/typecheck/should_fail/T8603.hs - + testsuite/tests/warnings/should_compile/RedundantCanonicalMethods.hs - + testsuite/tests/warnings/should_compile/RedundantCanonicalMethods.stderr - testsuite/tests/warnings/should_compile/WarnNoncanonical.hs - testsuite/tests/warnings/should_compile/WarnNoncanonical.stderr - testsuite/tests/warnings/should_compile/all.T - utils/check-exact/ExactPrint.hs - utils/check-ppr/Main.hs - utils/haddock/haddock-api/haddock-api.cabal - utils/haddock/haddock-api/src/Haddock/Backends/Hoogle.hs - utils/haddock/haddock-api/src/Haddock/Backends/LaTeX.hs - utils/haddock/haddock-api/src/Haddock/Backends/Xhtml/Decl.hs - utils/haddock/haddock-api/src/Haddock/Convert.hs - utils/haddock/haddock-api/src/Haddock/Interface.hs - utils/haddock/haddock-api/src/Haddock/Interface/ParseModuleHeader.hs - utils/haddock/haddock-api/src/Haddock/Interface/Rename.hs - utils/haddock/haddock-api/src/Haddock/Types.hs - utils/haddock/haddock-api/src/Haddock/Utils/Json.hs - utils/haddock/haddock-library/haddock-library.cabal - utils/haddock/haddock.cabal - utils/runghc/Main.hs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/1f6876c6402e886fe21a2582df1542… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/1f6876c6402e886fe21a2582df1542… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0

[Git][ghc/ghc][wip/marge_bot_batch_merge_job] 5 commits: X86 NCG: use 32-bit POPCNT for sub-word popcounts
by Marge Bot (＠marge-bot) 29 Oct '26

29 Oct '26

Marge Bot pushed to branch wip/marge_bot_batch_merge_job at Glasgow Haskell Compiler / GHC Commits: ac870ad5 by Simon Jakobi at 2026-10-28T04:40:42-04:00 X86 NCG: use 32-bit POPCNT for sub-word popcounts A 32-bit write zero-extends the result to the full register, so we don't need to emit additional code for this. We also skip emitting zero-extension instructions for the argument when it is already visibly zero-extended. For the byte-wise popcount loop from #24813, this shrinks the popcount sequence from 5 instructions to 3. Assisted-by: Claude Fable 5.1 - - - - - 53087373 by Facundo Domínguez at 2026-10-28T04:41:32-04:00 Use a ShortByteString in Serialized instead of [Word8] This makes the in-memory representation 40x more compact for all values stored in annotations. A test is added to measure the representation size of annotations specified in the compiler input. The GHC API is modified. All functions that previously handled annotations in [Word8] form now use ShortByteString. The following definitions have modified types: * GHC.findGlobalAnns * GHC.Tc.Gen.Splice.getAnnotationsByTypeRep * GHC.Core.Opt.Monad.getAnnotations * GHC.Core.Opt.Monad.getFirstAnnotations * GHC.Types.Annotations.deserializeAnns * GHC.Types.Annotations.findAnns * GHC.Types.Annotations.findAnnsByTypeRep * GHC.Serialized.Serialized * GHC.Serialized.toSerialized * GHC.Serialized.fromSerialized * GHC.Serialized.serializeWithData * GHC.Serialized.deserializeWithData * GHCi.Message.ReifyAnnotations Fixes #28002. - - - - - 20c35f20 by David Eichmann at 2026-10-28T22:26:22-04:00 testsuite: Fix python type errors These errors appear when using the VSCode python extension which uses Pylance under the hood. - - - - - 78803540 by David Eichmann at 2026-10-28T22:26:22-04:00 Add test modifier to test over all IO Managers - - - - - e3a21c96 by Alan Zimmerman at 2026-10-28T22:26:22-04:00 EPA: Tidy up ExactPrint.markAnnListA' It takes an action which can modify the AnnList, but this is never used. So remove it. - - - - - 30 changed files: - + changelog.d/T28002 - + changelog.d/T28002-api - + changelog.d/ncg-x86-subword-popcnt - compiler/GHC.hs - compiler/GHC/CmmToAsm/X86/CodeGen.hs - compiler/GHC/Core/Opt/Monad.hs - compiler/GHC/Tc/Gen/Splice.hs - compiler/GHC/Types/Annotations.hs - compiler/GHC/Utils/Outputable.hs - libraries/ghc-boot/GHC/Serialized.hs - libraries/ghci/GHCi/Message.hs - libraries/ghci/GHCi/TH.hs - libraries/ghci/GHCi/TH/Binary.hs - rts/RtsUtils.c - testsuite/config/ghc - testsuite/driver/runtests.py - testsuite/driver/testglobals.py - testsuite/driver/testlib.py - testsuite/driver/testutil.py - testsuite/tests/annotations/should_run/Makefile - + testsuite/tests/annotations/should_run/T16841.hs - + testsuite/tests/annotations/should_run/T16841.stdout - testsuite/tests/annotations/should_run/all.T - + testsuite/tests/codeGen/should_gen_asm/T24813.asm - + testsuite/tests/codeGen/should_gen_asm/T24813.cmm - testsuite/tests/codeGen/should_gen_asm/all.T - testsuite/tests/plugins/T26761Plugin.hs - testsuite/tests/plugins/all.T - testsuite/tests/rts/all.T - utils/check-exact/ExactPrint.hs Changes: ===================================== changelog.d/T28002 ===================================== @@ -0,0 +1,8 @@ +section: compiler +issues: #28002 +mrs: !16841 +synopsis: + Use ``ShortByteString`` in the ``Serialized`` type instead of ``[Word8]`` +description: + ``GHC.Serialized.Serialized`` now uses ``ByteString`` to store serialized data. + This makes in-memory representation more compact. ===================================== changelog.d/T28002-api ===================================== @@ -0,0 +1,21 @@ +section: ghc-lib +issues: #28002 +mrs: !16841 +synopsis: + Use ``ShortByteString`` in the ``Serialized`` type instead of ``[Word8]`` +description: + Breaking change: All functions that previously handled annotations in ``[Word8]`` + form now use ``ShortByteString``. The following definitions have modified types: + * ``GHC.findGlobalAnns`` + * ``GHC.Tc.Gen.Splice.getAnnotationsByTypeRep`` + * ``GHC.Core.Opt.Monad.getAnnotations`` + * ``GHC.Core.Opt.Monad.getFirstAnnotations`` + * ``GHC.Types.Annotations.deserializeAnns`` + * ``GHC.Types.Annotations.findAnns`` + * ``GHC.Types.Annotations.findAnnsByTypeRep`` + * ``GHC.Serialized.Serialized`` + * ``GHC.Serialized.toSerialized`` + * ``GHC.Serialized.fromSerialized`` + * ``GHC.Serialized.serializeWithData`` + * ``GHC.Serialized.deserializeWithData`` + * ``GHCi.Message.ReifyAnnotations`` ===================================== changelog.d/ncg-x86-subword-popcnt ===================================== @@ -0,0 +1,5 @@ +section: compiler +synopsis: The x86 native code generator now emits shorter code for 8-bit and + 16-bit ``popCount`` when :ghc-flag:`-msse4.2` is enabled. +mrs: !16640 +issues: #24813 ===================================== compiler/GHC.hs ===================================== @@ -453,12 +453,12 @@ import qualified GHC.Unit.Home.Graph as HUG import Control.Applicative ((<|>)) import Control.Monad import Control.Monad.Catch as MC +import Data.ByteString.Short (ShortByteString) import Data.Foldable import Data.Function ((&)) import Data.IORef import Data.List (isPrefixOf) import Data.Typeable ( Typeable ) -import Data.Word ( Word8 ) import qualified Data.Map.Strict as Map import Data.Set (Set) @@ -1526,7 +1526,7 @@ lookupGlobalName :: GhcMonad m => Name -> m (Maybe TyThing) lookupGlobalName name = withSession $ \hsc_env -> do liftIO $ lookupType hsc_env name -findGlobalAnns :: (GhcMonad m, Typeable a) => ([Word8] -> a) -> AnnTarget Name -> m [a] +findGlobalAnns :: (GhcMonad m, Typeable a) => (ShortByteString -> a) -> AnnTarget Name -> m [a] findGlobalAnns deserialize target = withSession $ \hsc_env -> do ann_env <- liftIO $ prepareAnnotations hsc_env Nothing return (findAnns deserialize ann_env target) ===================================== compiler/GHC/CmmToAsm/X86/CodeGen.hs ===================================== @@ -6739,20 +6739,23 @@ genPopCnt bid width dst src = do True -> do code_src <- getAnyReg src - src_r <- getNewRegNat format + -- Sub-word popcounts use 32-bit POPCNT, whose write zero-extends the + -- result to the full register. + src_r <- getNewRegNat (max II32 format) let dst_r = getRegisterReg platform (CmmLocal dst) + -- A sub-word argument is a full Word# whose upper bits may be + -- dirty, so zero-extend it unless it visibly already is. + zx_src + | CmmMachOp (MO_UU_Conv from _) [_] <- src, from <= width + = nilOL + | otherwise + = unitOL (MOVZxL format (OpReg src_r) (OpReg src_r)) return $ code_src src_r `appOL` - (if width == W8 then - -- The POPCNT instruction doesn't take a r/m8 - unitOL (MOVZxL II8 (OpReg src_r) (OpReg src_r)) `appOL` - unitOL (POPCNT II16 (OpReg src_r) dst_r) - else - unitOL (POPCNT format (OpReg src_r) dst_r)) `appOL` (if width == W8 || width == W16 then - -- We used a 16-bit destination register above, - -- so zero-extend - unitOL (MOVZxL II16 (OpReg dst_r) (OpReg dst_r)) - else nilOL) + zx_src `appOL` + unitOL (POPCNT II32 (OpReg src_r) dst_r) + else + unitOL (POPCNT format (OpReg src_r) dst_r)) False -> -- generate C call to hs_popcntN in ghc-prim ===================================== compiler/GHC/Core/Opt/Monad.hs ===================================== @@ -68,9 +68,9 @@ import GHC.Unit.Module.ModGuts import GHC.Unit.External import Data.Bifunctor ( bimap ) +import Data.ByteString.Short ( ShortByteString ) import Data.Dynamic import Data.Maybe (listToMaybe) -import Data.Word import Control.Monad import Control.Applicative ( Alternative(..) ) @@ -323,14 +323,14 @@ get_eps = do -- annotations. -- -- See Note [Annotations] -getAnnotations :: Typeable a => ([Word8] -> a) -> ModGuts -> CoreM (ModuleEnv [a], NameEnv [a]) +getAnnotations :: Typeable a => (ShortByteString -> a) -> ModGuts -> CoreM (ModuleEnv [a], NameEnv [a]) getAnnotations deserialize guts = do hsc_env <- getHscEnv ann_env <- liftIO $ prepareAnnotations hsc_env (Just guts) return (deserializeAnns deserialize ann_env) -- | Get at most one annotation of a given type per annotatable item. -getFirstAnnotations :: Typeable a => ([Word8] -> a) -> ModGuts -> CoreM (ModuleEnv a, NameEnv a) +getFirstAnnotations :: Typeable a => (ShortByteString -> a) -> ModGuts -> CoreM (ModuleEnv a, NameEnv a) getFirstAnnotations deserialize guts = bimap mod name <$> getAnnotations deserialize guts where ===================================== compiler/GHC/Tc/Gen/Splice.hs ===================================== @@ -155,10 +155,10 @@ import Control.DeepSeq import Control.Monad import Data.Binary import Data.Binary.Get +import Data.ByteString.Short (ShortByteString) import Data.Containers.ListUtils ( nubOrd ) import Data.List ( sortBy ) import Data.Maybe -import qualified Data.ByteString as B import qualified Data.ByteString.Lazy as LB import Data.Dynamic ( fromDynamic, toDyn ) import qualified Data.IntMap as IntMap @@ -1947,7 +1947,7 @@ handleTHMessage msg = case msg of ReifyInstances n ts -> wrapTHResult $ runQinTcM $ TH.reifyInstances n ts ReifyRoles n -> wrapTHResult $ runQinTcM $ TH.reifyRoles n ReifyAnnotations lookup tyrep -> - wrapTHResult $ (map B.pack <$> getAnnotationsByTypeRep lookup tyrep) + wrapTHResult $ getAnnotationsByTypeRep lookup tyrep ReifyModule m -> wrapTHResult $ runQinTcM $ TH.reifyModule m ReifyConStrictness nm -> wrapTHResult $ runQinTcM $ TH.reifyConStrictness nm GetPackageRoot -> wrapTHResult $ runQinTcM $ TH.getPackageRoot @@ -1967,7 +1967,7 @@ handleTHMessage msg = case msg of FailIfErrs -> wrapTHResult failIfErrsM _ -> panic ("handleTHMessage: unexpected message " ++ show msg) -getAnnotationsByTypeRep :: TH.AnnLookup -> TypeRep -> TcM [[Word8]] +getAnnotationsByTypeRep :: TH.AnnLookup -> TypeRep -> TcM [ShortByteString] getAnnotationsByTypeRep th_name tyrep = do { name <- lookupThAnnLookup th_name ; topEnv <- getTopEnv ===================================== compiler/GHC/Types/Annotations.hs ===================================== @@ -27,9 +27,9 @@ import GHC.Utils.Outputable import GHC.Serialized import Control.Monad +import Data.ByteString.Short ( ShortByteString ) import Data.Maybe import Data.Typeable -import Data.Word ( Word8 ) import Control.DeepSeq -- | Represents an annotation after it has been sufficiently desugared from @@ -112,14 +112,14 @@ plusAnnEnv a b = -- | Find the annotations attached to the given target as 'Typeable' -- values of your choice. If no deserializer is specified, -- only transient annotations will be returned. -findAnns :: Typeable a => ([Word8] -> a) -> AnnEnv -> CoreAnnTarget -> [a] +findAnns :: Typeable a => (ShortByteString -> a) -> AnnEnv -> CoreAnnTarget -> [a] findAnns deserialize env = mapMaybe (fromSerialized deserialize) . findAnnPayloads env -- | Find the annotations attached to the given target as 'Typeable' -- values of your choice. If no deserializer is specified, -- only transient annotations will be returned. -findAnnsByTypeRep :: AnnEnv -> CoreAnnTarget -> TypeRep -> [[Word8]] +findAnnsByTypeRep :: AnnEnv -> CoreAnnTarget -> TypeRep -> [ShortByteString] findAnnsByTypeRep env target tyrep = [ ws | Serialized tyrep' ws <- findAnnPayloads env target , tyrep' == tyrep ] @@ -134,7 +134,7 @@ findAnnPayloads env target = -- | Deserialize all annotations of a given type. This happens lazily, that is -- no deserialization will take place until the [a] is actually demanded and -- the [a] can also be empty (the UniqFM is not filtered). -deserializeAnns :: Typeable a => ([Word8] -> a) -> AnnEnv -> (ModuleEnv [a], NameEnv [a]) +deserializeAnns :: Typeable a => (ShortByteString -> a) -> AnnEnv -> (ModuleEnv [a], NameEnv [a]) deserializeAnns deserialize env = ( mapModuleEnv deserAnns (ann_mod_env env) , mapNameEnv deserAnns (ann_name_env env) ===================================== compiler/GHC/Utils/Outputable.hs ===================================== @@ -138,6 +138,7 @@ import GHC.Utils.Misc (lastMaybe, snocView) import Data.ByteString (ByteString) import qualified Data.ByteString as BS +import qualified Data.ByteString.Short as Short import Data.Char import qualified Data.Map as M import Data.Int @@ -1105,7 +1106,7 @@ instance Outputable a => Outputable (SCC a) where ppr (CyclicSCC vs) = text "REC" $$ (nest 3 (vcat (map ppr vs))) instance Outputable Serialized where - ppr (Serialized the_type bytes) = int (length bytes) <+> text "of type" <+> text (show the_type) + ppr (Serialized the_type bytes) = int (Short.length bytes) <+> text "of type" <+> text (show the_type) instance Outputable Extension where ppr = text . show ===================================== libraries/ghc-boot/GHC/Serialized.hs ===================================== @@ -19,144 +19,68 @@ module GHC.Serialized ( ) where import Prelude -- See note [Why do we import Prelude here?] -import Data.Bits -import Data.Word ( Word8 ) -import Data.Data import Control.DeepSeq +import Data.Binary +import Data.Binary.Get +import Data.Binary.Put +import Data.ByteString.Short (ShortByteString) +import qualified Data.ByteString.Short as Short +import qualified Data.ByteString.Lazy as ByteString.Lazy +import Data.Data -- | Represents a serialized value of a particular type. Attempts can be made to deserialize it at certain types -data Serialized = Serialized TypeRep [Word8] +data Serialized = Serialized TypeRep ShortByteString instance NFData Serialized where rnf (Serialized tr ws) = rnf tr `seq` rnf ws -- | Put a Typeable value that we are able to actually turn into bytes into a 'Serialized' value ready for deserialization later -toSerialized :: forall a. Typeable a => (a -> [Word8]) -> a -> Serialized +toSerialized :: forall a. Typeable a => (a -> ShortByteString) -> a -> Serialized toSerialized serialize what = Serialized (typeOf what) (serialize what) -- | If the 'Serialized' value contains something of the given type, then use the specified deserializer to return @Just@ that. -- Otherwise return @Nothing@. -fromSerialized :: forall a. Typeable a => ([Word8] -> a) -> Serialized -> Maybe a +fromSerialized :: forall a. Typeable a => (ShortByteString -> a) -> Serialized -> Maybe a fromSerialized deserialize (Serialized the_type bytes) | the_type == rep = Just (deserialize bytes) | otherwise = Nothing where rep = typeRep (Proxy :: Proxy a) -- | Use a 'Data' instance to implement a serialization scheme dual to that of 'deserializeWithData' -serializeWithData :: Data a => a -> [Word8] -serializeWithData what = serializeWithData' what [] - -serializeWithData' :: Data a => a -> [Word8] -> [Word8] -serializeWithData' what = fst $ gfoldl (\(before, a_to_b) a -> (before . serializeWithData' a, a_to_b a)) - (\x -> (serializeConstr (constrRep (toConstr what)), x)) - what +serializeWithData :: Data a => a -> ShortByteString +serializeWithData = Short.toShort . ByteString.Lazy.toStrict . runPut . putData -- | Use a 'Data' instance to implement a deserialization scheme dual to that of 'serializeWithData' -deserializeWithData :: Data a => [Word8] -> a -deserializeWithData = snd . deserializeWithData' - -deserializeWithData' :: forall a. Data a => [Word8] -> ([Word8], a) -deserializeWithData' bytes = deserializeConstr bytes $ \constr_rep bytes -> - gunfold (\(bytes, b_to_r) -> let (bytes', b) = deserializeWithData' bytes in (bytes', b_to_r b)) - (\x -> (bytes, x)) - (repConstr (dataTypeOf (undefined :: a)) constr_rep) - - -serializeConstr :: ConstrRep -> [Word8] -> [Word8] -serializeConstr (AlgConstr ix) = serializeWord8 1 . serializeInt ix -serializeConstr (IntConstr i) = serializeWord8 2 . serializeInteger i -serializeConstr (FloatConstr r) = serializeWord8 3 . serializeRational r -serializeConstr (CharConstr c) = serializeWord8 4 . serializeChar c - - -deserializeConstr :: [Word8] -> (ConstrRep -> [Word8] -> a) -> a -deserializeConstr bytes k = deserializeWord8 bytes $ \constr_ix bytes -> - case constr_ix of - 1 -> deserializeInt bytes $ \ix -> k (AlgConstr ix) - 2 -> deserializeInteger bytes $ \i -> k (IntConstr i) - 3 -> deserializeRational bytes $ \r -> k (FloatConstr r) - 4 -> deserializeChar bytes $ \c -> k (CharConstr c) - x -> error $ "deserializeConstr: unrecognised serialized constructor type " ++ show x ++ " in context " ++ show bytes - - -serializeFixedWidthNum :: forall a. (Integral a, FiniteBits a) => a -> [Word8] -> [Word8] -serializeFixedWidthNum what = go (finiteBitSize what) what - where - go :: Int -> a -> [Word8] -> [Word8] - go size current rest - | size <= 0 = rest - | otherwise = fromIntegral (current .&. 255) : go (size - 8) (current `shiftR` 8) rest - -deserializeFixedWidthNum :: forall a b. (Integral a, FiniteBits a) => [Word8] -> (a -> [Word8] -> b) -> b -deserializeFixedWidthNum bytes k = go (finiteBitSize (undefined :: a)) bytes k - where - go :: Int -> [Word8] -> (a -> [Word8] -> b) -> b - go size bytes k - | size <= 0 = k 0 bytes - | otherwise = case bytes of - (byte:bytes) -> go (size - 8) bytes (\x -> k ((x `shiftL` 8) .|. fromIntegral byte)) - [] -> error "deserializeFixedWidthNum: unexpected end of stream" - - -serializeEnum :: (Enum a) => a -> [Word8] -> [Word8] -serializeEnum = serializeInt . fromEnum - -deserializeEnum :: Enum a => [Word8] -> (a -> [Word8] -> b) -> b -deserializeEnum bytes k = deserializeInt bytes (k . toEnum) - - -serializeWord8 :: Word8 -> [Word8] -> [Word8] -serializeWord8 x = (x:) - -deserializeWord8 :: [Word8] -> (Word8 -> [Word8] -> a) -> a -deserializeWord8 (byte:bytes) k = k byte bytes -deserializeWord8 [] _ = error "deserializeWord8: unexpected end of stream" - - -serializeInt :: Int -> [Word8] -> [Word8] -serializeInt = serializeFixedWidthNum - -deserializeInt :: [Word8] -> (Int -> [Word8] -> a) -> a -deserializeInt = deserializeFixedWidthNum - - -serializeRational :: (Real a) => a -> [Word8] -> [Word8] -serializeRational = serializeString . show . toRational - -deserializeRational :: (Fractional a) => [Word8] -> (a -> [Word8] -> b) -> b -deserializeRational bytes k = deserializeString bytes (k . fromRational . read) - - -serializeInteger :: Integer -> [Word8] -> [Word8] -serializeInteger = serializeString . show - -deserializeInteger :: [Word8] -> (Integer -> [Word8] -> a) -> a -deserializeInteger bytes k = deserializeString bytes (k . read) - - -serializeChar :: Char -> [Word8] -> [Word8] -serializeChar = serializeString . show - -deserializeChar :: [Word8] -> (Char -> [Word8] -> a) -> a -deserializeChar bytes k = deserializeString bytes (k . read) - - -serializeString :: String -> [Word8] -> [Word8] -serializeString = serializeList serializeEnum - -deserializeString :: [Word8] -> (String -> [Word8] -> a) -> a -deserializeString = deserializeList deserializeEnum - - -serializeList :: (a -> [Word8] -> [Word8]) -> [a] -> [Word8] -> [Word8] -serializeList serialize_element xs = serializeInt (length xs) . foldr (.) id (map serialize_element xs) - -deserializeList :: forall a b. (forall c. [Word8] -> (a -> [Word8] -> c) -> c) - -> [Word8] -> ([a] -> [Word8] -> b) -> b -deserializeList deserialize_element bytes k = deserializeInt bytes $ \len bytes -> go len bytes k - where - go :: Int -> [Word8] -> ([a] -> [Word8] -> b) -> b - go len bytes k - | len <= 0 = k [] bytes - | otherwise = deserialize_element bytes (\elt bytes -> go (len - 1) bytes (k . (elt:))) +deserializeWithData :: Data a => ShortByteString -> a +deserializeWithData bs = + case runGetOrFail getData (ByteString.Lazy.fromStrict (Short.fromShort bs)) of + Right (_, _, x) -> x + Left (_, off, msg) -> + error ("deserializeWithData: " ++ msg ++ " at offset " ++ show off) + +putData :: Data a => a -> Put +putData x = do + putConstrRep (constrRep (toConstr x)) + sequence_ (gmapQ putData x) + +getData :: forall a. Data a => Get a +getData = do + rep <- getConstrRep + fromConstrM getData (repConstr (dataTypeOf (undefined :: a)) rep) + +putConstrRep :: ConstrRep -> Put +putConstrRep (AlgConstr ix) = putWord8 1 >> put ix +putConstrRep (IntConstr i) = putWord8 2 >> put i +putConstrRep (FloatConstr r) = putWord8 3 >> put r +putConstrRep (CharConstr c) = putWord8 4 >> put c + +getConstrRep :: Get ConstrRep +getConstrRep = do + tag <- getWord8 + case tag of + 1 -> AlgConstr <$> get + 2 -> IntConstr <$> get + 3 -> FloatConstr <$> get + 4 -> CharConstr <$> get + _ -> fail ("unrecognised constructor tag " ++ show tag) ===================================== libraries/ghci/GHCi/Message.hs ===================================== @@ -297,7 +297,7 @@ data THMessage a where ReifyInstances :: TH.Name -> [TH.Type] -> THMessage (THResult [TH.Dec]) ReifyRoles :: TH.Name -> THMessage (THResult [TH.Role]) ReifyAnnotations :: TH.AnnLookup -> TypeRep - -> THMessage (THResult [ByteString]) + -> THMessage (THResult [BS.ShortByteString]) ReifyModule :: TH.Module -> THMessage (THResult TH.ModuleInfo) ReifyConStrictness :: TH.Name -> THMessage (THResult [TH.DecidedStrictness]) ===================================== libraries/ghci/GHCi/TH.hs ===================================== @@ -100,7 +100,6 @@ import Control.Monad.IO.Class (MonadIO (..)) import Data.Binary import Data.Binary.Put import Data.ByteString (ByteString) -import qualified Data.ByteString as B import qualified Data.ByteString.Lazy as LB import Data.Data import Data.Dynamic @@ -169,7 +168,7 @@ instance MonadIO GHCiQ where -- serialize irrelevant annotations. reifyAnnotations :: forall a . Data a => TH.AnnLookup -> GHCiQ [a] reifyAnnotations lookup = - map (deserializeWithData . B.unpack) <$> + map deserializeWithData <$> ghcCmd (ReifyAnnotations lookup typerep) where typerep = typeOf (undefined :: a) ===================================== libraries/ghci/GHCi/TH/Binary.hs ===================================== @@ -9,7 +9,6 @@ module GHCi.TH.Binary () where import Prelude -- See note [Why do we import Prelude here?] import Data.Binary -import qualified Data.ByteString as B import qualified Data.ByteString.Internal as B import GHC.Serialized import qualified GHC.Boot.TH.Syntax as TH @@ -74,8 +73,8 @@ instance Binary TH.DocLoc -- We need Binary TypeRep for serializing annotations instance Binary Serialized where - put (Serialized tyrep wds) = put tyrep >> put (B.pack wds) - get = Serialized <$> get <*> (B.unpack <$> get) + put (Serialized tyrep wds) = put tyrep >> put wds + get = Serialized <$> get <*> get instance Binary TH.Bytes where put (TH.Bytes ptr off sz) = put bs ===================================== rts/RtsUtils.c ===================================== @@ -397,6 +397,7 @@ void printRtsInfo(const RtsConfig rts_config) { rts_config.rts_opts != NULL ? rts_config.rts_opts : ""); selectIOManager(); /* resolve the io-manager, accounting for flags */ mkRtsInfoPair("I/O manager default", showIOManager()); + mkRtsInfoPair("I/O managers", &IOMGRS_ENABLED_STR[1]); // skips the initial ' ' character in IOMGRS_ENABLED_STR printf(" ]\n"); } ===================================== testsuite/config/ghc ===================================== @@ -257,7 +257,9 @@ def get_compiler_info(): ) except Exception as err: - print("Exception thrown in testsuite/config/ghc.get_compiler_info: %s", err) + # We don't throw so that we can still discover tests before we've built + # GHC (i.e. config.compiler doesn't yet exist). + print("Exception thrown in testsuite/config/ghc.get_compiler_info: ", err) else: res = p.returncode == 0 @@ -315,6 +317,58 @@ def get_compiler_info(): config.ghc_th_way = "normal" config.ghc_th_hi_ext = "hi" + # I/O manager discovery. + # + # Which I/O managers (GHC's `--io-manager` RTS flag) a given RTS build + # supports depends on the target platform and on whether the RTS is the + # threaded or non-threaded flavour (see rts/IOManager.h's + # IOMGR_ENABLED_* macros). Rather than duplicate that platform/way + # dependent logic here, we ask the RTS itself using `+RTS --info -RTS` + # and extract the "I/O managers" value from stdout. + def discover_io_managers_generic(extra_flags): + import tempfile + + test_src = 'main :: IO ()\nmain = return ()\n' + + res = [None] + + with tempfile.TemporaryDirectory() as d: + src = Path(d) / 'IOMgrProbe.hs' + src.write_text(test_src) + exe = Path(d) / ('IoMgrProbe' + config.exeext) + + # -rtsopts is needed so that the resulting binary will even + # consider an --io-manager= flag (rather than immediately + # rejecting all "unsafe" RTS options). + cmd = '{} -v0 -rtsopts -o {} {} {}'.format( + config.compiler, + exe, + src, + ' '.join(extra_flags)) + try: + cp = subprocess.run( + cmd, + shell=True, + cwd=d, + stdout=subprocess.DEVNULL, + stderr=None if config.verbose >= 3 else subprocess.DEVNULL) + if cp.returncode != 0 or not exe.exists(): + raise Exception("Error in testsuite/config/ghc.discover_io_managers: failed to compile with: {}".format(cmd)) + res = list(discover_io_managers(exe, d)) + + except Exception as err: + # We don't throw so that we can still discover tests before we've built + # GHC (i.e. config.compiler doesn't yet exist). + print("Exception thrown in testsuite/config/ghc.discover_io_managers_generic: ", err) + + return res + + config.io_managers_nonthreaded = discover_io_managers_generic([]) + config.io_managers_threaded = ( + discover_io_managers_generic(['-threaded']) if config.ghc_with_threaded_rts + else []) + config.io_managers_all = list(set(config.io_managers_nonthreaded).union(set(config.io_managers_threaded))) + config.cmm_cpp_is_gcc = test_compile( ['-c'], test_filename='test.cmm', ===================================== testsuite/driver/runtests.py ===================================== @@ -28,7 +28,7 @@ import subprocess import asyncio -from testutil import getStdout, str_warn, str_info, print_table, shorten_metric_name, str_removeprefix +from testutil import discover_io_managers, getStdout, str_warn, str_info, print_table, shorten_metric_name, str_removeprefix from testglobals import getConfig, ghc_env, TestConfig, t, \ TestOptions, brokens, PerfMetric from my_typing import TestName ===================================== testsuite/driver/testglobals.py ===================================== @@ -126,6 +126,15 @@ class TestConfig: self.way_flags = {} # type: Dict[WayName, List[str]] self.way_rts_flags = {} # type: Dict[WayName, List[str]] + # I/O managers (GHC's `--io-manager` RTS flag) supported by the threaded runtime. + self.io_managers_threaded = [] # type: List[str] + + # I/O managers (GHC's `--io-manager` RTS flag) supported by the non-threaded runtime. + self.io_managers_nonthreaded = [] # type: List[str] + + # All I/O managers (GHC's `--io-manager` RTS flag) + self.io_managers_all = [] # type: List[str] + # Do we have a functional LLVM toolchain? self.have_llvm = False @@ -227,7 +236,7 @@ class TestConfig: self.allowed_perf_changes = {} # The test environment. - self.test_env = 'local' + self.test_env = TestEnv('local') # terminal supports colors self.supports_colors = False @@ -367,6 +376,14 @@ class TestOptions: # add these ways to the default set self.extra_ways = [] # type: List[WayName] + # IO Managers to test (passed as the --io-manager RTS option) + # * [None] - only default IO Manager + # * [...] - list of io managers e.g. "mio" or "poll" + self.io_managers = [None] # type: List[Optional[str]] + + # IO Manager used in the current test case. Set before each test case. None means default IO Manager + self.io_manager_current = None # type: Optional[str] + # the result we normally expect for this test self.expect = 'pass' ===================================== testsuite/driver/testlib.py ===================================== @@ -23,7 +23,7 @@ import subprocess from testglobals import config, ghc_env, default_testopts, brokens, t, \ TestRun, TestResult, TestOptions, PerfMetric -from testutil import strip_quotes, lndir, link_or_copy_file, passed, \ +from testutil import discover_io_managers, strip_quotes, lndir, link_or_copy_file, passed, \ failBecause, residency_testing_metrics, \ stable_perf_counters, \ PassFail, badResult, str_warn, str_removeprefix @@ -190,7 +190,7 @@ def no_lint( name, opts ): def stage1(name, opts): # See Note [Why is there no stage1 setup function?] - framework_fail(name, 'stage1 setup function does not exist', + framework_fail(name, None, 'add your test to testsuite/tests/stage1 instead') # Note [Why is there no stage1 setup function?] @@ -341,7 +341,7 @@ def req_th( name, opts ): req_interp(name, opts); if ghc_dynamic(): - return _omit_ways(name, opts, ['profasm', 'profthreaded']) + return _omit_ways(name, opts, [WayName('profasm'), WayName('profthreaded')]) def req_plugins( name, opts ): """ @@ -355,7 +355,7 @@ def req_plugins( name, opts ): req_interp(name, opts) # Plugins aren't supported with the external interpreter (#14335) - expect_broken_for(14335,['ext-interp'])(name,opts) + expect_broken_for(IssueNumber(14335),[WayName('ext-interp')])(name,opts) if config.cross: opts.skip = True @@ -569,6 +569,34 @@ def extra_ways( ways: List[WayName] ): return helper +# ----- + +def _io_managers( name: TestName, opts, io_mgrs: List[str] ) -> None: + opts.io_managers = io_mgrs + +def io_managers( io_mgrs: List[str] ): + """ + Further split test cases into the given IO Managers if available. + Available IO Managers are determined by the test case's way (threaded/non-threads) and OS. + You'll likely want to use `extra_ways(['threaded2'])` if you want to test threaded IO Managers (e.g. mio). + """ + return lambda name, opts: _io_managers(name, opts, io_mgrs) + +def all_available_io_managers( name: TestName, opts ): + """ + Further split test cases into all available IO Managers. + Available IO Managers are determined by the test case's way (threaded/non-threads) and OS. + You'll likely want to use `extra_ways(['threaded2'])` to ensure threaded IO Managers are tested (e.g. mio). + """ + _io_managers(name, opts, config.io_managers_all) + +def all_available_io_managers_except(exclude: List[str]): + """ + Further split test cases into all available (except the given excluded) IO Managers. + Available IO Managers are determined by the test case's way (threaded/non-threads) and OS. + You'll likely want to use `extra_ways(['threaded2'])` to ensure threaded IO Managers are tested (e.g. mio). + """ + return lambda name, opts: _io_managers(name, opts, list(set(config.io_managers_all) - set(exclude))) # ----- @@ -661,18 +689,15 @@ def collect_size ( deviation, path ): def collect_size_func ( deviation, path_func ): return collect_generic_stat ( 'size', deviation, lambda way: os.path.getsize(in_testdir(path_func())) ) -def get_dir_size(path): +def get_dir_size(path) -> int: total = 0 - try: - with os.scandir(path) as it: - for entry in it: - if entry.is_file(): - total += entry.stat().st_size - elif entry.is_dir(): - total += get_dir_size(entry.path) - return total - except FileNotFoundError: - print("Exception: Could not find: " + path) + with os.scandir(path) as it: + for entry in it: + if entry.is_file(): + total += entry.stat().st_size + elif entry.is_dir(): + total += get_dir_size(entry.path) + return total def collect_size_dir ( deviation, path ): return collect_size_dir_func ( deviation, lambda: path ) @@ -790,7 +815,7 @@ def find_non_inplace_so(lib): return _find_so(lib,path_from_ghcPkg(lib, "dynamic-library-dirs"),False) -def collect_generic_stat ( metric, deviation: Optional[int], get_stat: Callable[[WayName], str]): +def collect_generic_stat ( metric, deviation: Optional[int], get_stat: Callable[[WayName], Any]): """ Define a generic stat test, which computes the statistic by calling the function given as the third argument. @@ -1135,7 +1160,7 @@ def _collect_compiler_perf_counters(counters: Set[str], deviation: Optional[int] # Slightly hacky, we need the requested perf_counters in 'simple_run'. # Thus, we have to globally register these counters opts.compiler_perf_counters += list(counters) - _collect_stats(name, opts, counters, deviation, False, True, True) + _collect_stats(name, opts, counters, deviation, None, True, True) return f @@ -1334,21 +1359,22 @@ def check_errmsg(needle): # grep_errmsg(regex,[groups]) # If groups are given, return only the matched groups # that matches the regex. -def grep_errmsg_norm(needle:str, groups = None): - - def get_match(str:str): - m = re.search(needle,str) - if m: - return "".join([m.group(g) for g in groups if m.group(g) is not None]) - else: - return None +def grep_errmsg_norm(needle:str, groups_may = None): - def norm(str) -> str: - if groups == None: + def norm(string) -> str: + if groups_may == None: return "".join( filter(lambda l: re.search(needle,l), - str.splitlines(True))) + string.splitlines(True))) else: - matches = [get_match(x) for x in str.splitlines(True)] + groups = groups_may + def get_match(string: str): + m = re.search(needle, string) + if m: + return "".join([m.group(g) for g in groups if m.group(g) is not None]) + else: + return None + + matches = [get_match(x) for x in string.splitlines(True)] res = "\n".join([x for x in matches if x]) return res return norm @@ -1573,6 +1599,15 @@ do_not_copy = ('.hi', '.o', '.dyn_hi' , '.dyn_o', '.out' ,'.hi-boot', '.o-boot') # 12112 +def way_supports_io_manager(way: WayName, io_manager: Optional[str]) -> bool: + # All ways support a default IO Manager + if io_manager is None: + return True + + way_is_threaded = '-threaded' in config.way_flags[way] + available_io_managers = config.io_managers_threaded if way_is_threaded else config.io_managers_nonthreaded + return io_manager in available_io_managers + async def test_common_work(name: TestName, opts, func, args) -> None: try: @@ -1621,7 +1656,22 @@ async def test_common_work(name: TestName, opts, # A test itself can request extra ways by setting opts.extra_ways all_ways = list(OrderedDict.fromkeys(all_ways + [way for way in opts.extra_ways if way not in all_ways])) - t.total_test_cases += len(all_ways) + # We split each way over available io managers + if len(opts.io_managers) == 0: + framework_fail(name, None, 'opts.io_managers is empty') + all_ways_and_io_managers = dict( + ( + way, + [io_manager + for io_manager in opts.io_managers + if way_supports_io_manager(way, io_manager) + ] + ) + for way in all_ways + ) + + for io_managers in all_ways_and_io_managers.values(): + t.total_test_cases += len(io_managers) only_ways = getTestOpts().only_ways ok_way = lambda way: \ @@ -1704,19 +1754,23 @@ async def test_common_work(name: TestName, opts, # Run the required tests... for way in do_ways: - if stopping(): - break - start = time.monotonic() - try: - await do_test(name, way, func, args, files) - except KeyboardInterrupt: - stopNow() - except Exception as e: - traceback.print_exc() - framework_fail(name, way, traceback.format_exc(), - runtime=time.monotonic() - start) + for io_manager in all_ways_and_io_managers[way]: + if stopping(): + break + start = time.monotonic() + try: + getTestOpts().io_manager_current = io_manager + await do_test(name ,way ,func ,args ,files) + except KeyboardInterrupt: + stopNow() + except Exception as e: + traceback.print_exc() + framework_fail(name, way, traceback.format_exc(), + runtime=time.monotonic() - start) + + for skipped_way in set(all_ways) - set(do_ways): + t.n_tests_skipped += len(all_ways_and_io_managers[skipped_way]) - t.n_tests_skipped += len(set(all_ways) - set(do_ways)) if getTestOpts().expect == 'missing-lib': t.n_missing_libs += 1 if config.cleanup and do_ways: @@ -1743,6 +1797,8 @@ async def do_test(name: TestName, start = time.monotonic() full_name = name + '(' + way + ')' + if opts.io_manager_current is not None: + full_name += ' (IO Manager: {})'.format(opts.io_manager_current) test_n = len(allTestNames) progress_args = [ full_name, t.total_tests, test_n, [len(t.unexpected_passes), @@ -1925,7 +1981,7 @@ def framework_warn(name: TestName, way: WayName, reason: str) -> None: # run_command. async def run_command( name, way, cmd ): - return await simple_run( name, '', override_options(cmd), '' ) + return await simple_run( name, WayName(''), override_options(cmd), '' ) async def makefile_test( name, way, target=None ): if target is None: @@ -2011,7 +2067,7 @@ async def multi_compile_fail( name, way, top_mod, extra_mods, extra_hc_opts ): return await do_compile( name, way, True, top_mod, extra_mods, [], extra_hc_opts) async def make_depend( name, way, mods, extra_hc_opts ): - return await do_compile( name, way, False, ' '.join(mods), [], [], extra_hc_opts, mode = '-M') + return await do_compile( name, way, False, Path(' '.join(mods)), [], [], extra_hc_opts, mode = '-M') async def do_compile(name: TestName, way: WayName, @@ -2142,7 +2198,7 @@ async def compile_grep_core(name: TestName, async def compile_and_run__(name: TestName, way: WayName, - top_mod: Path, + top_mod: Optional[Path], extra_mods: List[str], extra_hc_opts: str, backpack: bool=False, @@ -2425,6 +2481,9 @@ async def simple_run(name: TestName, way: WayName, prog: str, extra_run_opts: st my_rts_flags = rts_flags(way) + if opts.io_manager_current is not None: + my_rts_flags += ' +RTS --io-manager={0} -RTS '.format(opts.io_manager_current) + # Collect runtime stats if necessary: # isStatsTest and not isCompilerStatsTest(): # assume we are running a ghc compiled program. Collect stats. @@ -2450,6 +2509,12 @@ async def simple_run(name: TestName, way: WayName, prog: str, extra_run_opts: st output_file = perfStatsFile(False, name) + # Check that IO Manager is available + if opts.io_manager_current is not None: + available_io_managers = discover_io_managers(prog, opts.testdir) + if opts.io_manager_current not in available_io_managers: + framework_fail(name, way, 'IO Manager {} unexpectedly unavailable (available IO Managers: {}). `-non-threaded`/`-threaded` is only supported through way RTS flags (not manually added via test modifiers/args) when testing over IO Managers.'.format(opts.io_manager_current, available_io_managers)) + # run the command exit_code = await runCmdPerf(opts.compiler_perf_counters, cmd, output_file, working_dir=opts.testdir, @@ -2511,7 +2576,7 @@ def rts_flags(way: WayName) -> str: async def interpreter_run(name: TestName, way: WayName, extra_hc_opts: str, - top_mod: Path + top_mod: Optional[Path] ) -> PassFail: opts = getTestOpts() @@ -2882,11 +2947,11 @@ async def compare_outputs( if config.accept_platform: if_verbose(1, 'Accepting new output for platform "' + config.platform + '".') - expected_path += '-' + config.platform + expected_path = Path(str(expected_path) + '-' + config.platform) elif config.accept_os: if_verbose(1, 'Accepting new output for os "' + config.os + '".') - expected_path += '-' + config.os + expected_path = Path(str(expected_path) + '-' + config.os) else: if_verbose(1, 'Accepting new output.') @@ -3516,7 +3581,7 @@ def find_expected_file(name: TestName, suff: str, way: WayName) -> Path: for f in files: if in_srcdir(f).exists(): - return f + return Path(f) return basename ===================================== testsuite/driver/testutil.py ===================================== @@ -1,4 +1,5 @@ import os +import re import subprocess import shutil import tempfile @@ -179,3 +180,19 @@ def shorten_metric_name(name: str) -> str: "binary_size/if_compression(3)": "size/3", } return dic.get(name, name) + +# I/O manager discovery for a haskell binary. Returns the set of IO Manager +# strings supported by the binary +def discover_io_managers(hs_exe: Union[str, Path], cwd: Union[str, Path]) -> Set[str]: + rp = subprocess.run( + [str(hs_exe), '+RTS', '--info', '-RTS'], + cwd=cwd, + stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL) + stdout = rp.stdout.decode('utf-8', errors='replace') + + m = re.search(r'$"I/O managers", "(.*)"$', stdout) + if not m: + raise Exception('Expected stdout output of form \'... ("I/O managers", "select selectbis poll") ...\' but got:\n{}'.format(stdout)) + + return set(m.group(1).split()) ===================================== testsuite/tests/annotations/should_run/Makefile ===================================== @@ -9,3 +9,7 @@ config : echo "module Config where" >>$(CONFIG_HS) echo "cTop :: String" >> $(CONFIG_HS) echo 'cTop = "$(subst \,\\,$(shell '$(TEST_HC)' --print-libdir))"' >> $(CONFIG_HS) + +T16841: + '$(TEST_HC)' $(TEST_HC_OPTS) -v0 -c T16841.hs + '$(TEST_HC)' $(TEST_HC_OPTS) --show-iface T16841.hi | grep 'annotated by' ===================================== testsuite/tests/annotations/should_run/T16841.hs ===================================== @@ -0,0 +1,15 @@ +-- | Measures the overhead of serializing annotations of a few types. +module T16841 where + +foo :: Int +foo = 42 + +{-# ANN foo (42 :: Int) #-} +{-# ANN foo (2^100 :: Integer) #-} +{-# ANN foo (3.25 :: Double) #-} +{-# ANN foo 'x' #-} +{-# ANN foo "helloworld" #-} +{-# ANN foo (Just True) #-} +{-# ANN foo ([1..100] :: [Int]) #-} +{-# ANN module (Left 7 :: Either Int String) #-} + ===================================== testsuite/tests/annotations/should_run/T16841.stdout ===================================== @@ -0,0 +1,9 @@ +Module target T16841 annotated by 15 of type Either Int [Char] +Named target foo annotated by 1509 of type [Int] +Named target foo annotated by 18 of type Maybe Bool +Named target foo annotated by 119 of type [Char] +Named target foo annotated by 2 of type Char +Named target foo annotated by 11 of type Double +Named target foo annotated by 24 of type Integer +Named target foo annotated by 6 of type Int + ===================================== testsuite/tests/annotations/should_run/all.T ===================================== @@ -16,6 +16,10 @@ test('annrun01', multimod_compile_and_run, ['annrun01', '-package ghc ' + config.ghc_th_way_flags]) +# Skipping on windows because of platform-specific failures. See +# https://gitlab.haskell.org/ghc/ghc/-/merge_requests/16841#note_706770 +test('T16841', [req_th, windows_skip, extra_files(['T16841.hs'])], makefile_test, ['T16841']) + """" Helpful things to C+P: ===================================== testsuite/tests/codeGen/should_gen_asm/T24813.asm ===================================== @@ -0,0 +1,10 @@ +popcnt8: + movzbl + popcnt + movq + jmp +popcnt16: + movzwl + popcnt + movq + jmp ===================================== testsuite/tests/codeGen/should_gen_asm/T24813.cmm ===================================== @@ -0,0 +1,16 @@ +#include "Cmm.h" + +// Popcount of a zero-extended sub-word value should be a single +// zero-extending load followed by one POPCNT (#24813). + +popcnt8 (W_ p) { + W_ r; + (r) = prim %popcnt8(%zx64(bits8[p])); + return (r); +} + +popcnt16 (W_ p) { + W_ r; + (r) = prim %popcnt16(%zx64(bits16[p])); + return (r); +} ===================================== testsuite/tests/codeGen/should_gen_asm/all.T ===================================== @@ -27,6 +27,7 @@ test('T25233', [unless(arch('x86_64'), skip), when(unregisterised(), skip)], compile_grep_asm, ['hs', True, '-O']) test('T25233b', [unless(arch('x86_64'), skip), when(unregisterised(), skip)], compile_grep_asm, ['cmm', True, '']) +test('T24813', is_amd64_codegen, compile_cmp_asm, ['cmm', '-msse4.2']) is_aarch64_codegen = [ unless(arch('aarch64'), skip), when(unregisterised(), skip), ===================================== testsuite/tests/plugins/T26761Plugin.hs ===================================== @@ -7,6 +7,7 @@ import GHC import GHC.Plugins import GHC.Tc.Types import GHC.Tc.Utils.Monad +import qualified Data.ByteString.Short as Short import qualified Data.Map as Map import Data.Word (Word8) import System.IO (hFlush, stdout) @@ -38,5 +39,5 @@ plugin = defaultPlugin { typeCheckResultAction = typecheckPlugin } -- check that it is visible from other modules. let ann = Annotation (ModuleTarget $ tcg_mod gblEnv) - (toSerialized id [0]) + (toSerialized id (Short.pack [0])) return gblEnv { tcg_anns = ann : tcg_anns gblEnv } ===================================== testsuite/tests/plugins/all.T ===================================== @@ -155,7 +155,7 @@ test('T14335', -fexternal-interpreter -package simple-plugin ' + config.plugin_way_flags]) test('T26761', [extra_files(['T26761Plugin.hs', 'T26761a.hs'])], - multimod_compile, ['T26761', '-package ghc ' + config.plugin_way_flags ]) + multimod_compile, ['T26761', '-package ghc -package bytestring ' + config.plugin_way_flags ]) test('plugin-recomp-pure', [extra_files(['plugin-recomp/', 'plugin-recomp-test.hs']), ===================================== testsuite/tests/rts/all.T ===================================== @@ -674,8 +674,9 @@ test('T23142', [unless(debug_rts(), skip), req_interp], makefile_test, ['T23142' test('T23400', [], compile_and_run, ['-with-rtsopts -A8k']) test('IOManager', [js_skip, when(arch('wasm32'), skip), when(opsys('mingw32'), skip), - pre_cmd('$MAKE -s --no-print-directory IOManager.hs')], - compile_and_run, ['']) + pre_cmd('$MAKE -s --no-print-directory IOManager.hs'), + extra_ways(['threaded2']), all_available_io_managers_except('winio')], + compile_and_run, ['']) test('T24142', [req_target_smp], compile_and_run, ['-threaded -with-rtsopts "-N2"']) ===================================== utils/check-exact/ExactPrint.hs ===================================== @@ -1486,9 +1486,7 @@ markAnnList ann action = do markAnnList' :: (Monad m, Monoid w) => AnnList -> EP w m a -> EP w m (AnnList, a) markAnnList' ann action = do - markAnnListA' ann $ \a -> do - r <- action - return (a,r) + markAnnListA' ann $ action markAnnListA :: (Monad m, Monoid w) => EpAnn AnnList @@ -1503,14 +1501,14 @@ markAnnListA an action = do markAnnListA' :: (Monad m, Monoid w) => AnnList - -> (AnnList -> EP w m (AnnList, a)) + -> EP w m a -> EP w m (AnnList, a) markAnnListA' an action = do an0 <- markLensBracketsO' an lal_brackets an1 <- markEpAnnAllLT' an0 lal_semis - (an2, r) <- action an1 - an3 <- markLensBracketsC' an2 lal_brackets - return (an3, r) + r <- action + an2 <- markLensBracketsC' an1 lal_brackets + return (an2, r) -- --------------------------------------------------------------------- @@ -2413,9 +2411,7 @@ instance ExactPrint (ClsInstDecl GhcPs) where , cid_modifiers = mods }) = do (mbWarn', i', w', mbOverlap', inst_ty', mods') <- top_matter - (ann_list', decls') <- markAnnListA' ann_list $ \a -> do - r <- setLayoutBoth $ mapM markAnnotated decls - return (a,r) + (ann_list', decls') <- markAnnListA' ann_list $ setLayoutBoth $ mapM markAnnotated decls return (ClsInstDecl { cid_ext = (mbWarn', AnnClsInstDecl i' w' ann_list') , cid_poly_ty = inst_ty' , cid_decls = decls' @@ -3336,9 +3332,7 @@ markMaybeDodgyStmts (an,l) stmts = c' <- markEpToken c return (Left (o',c'), r) Right an' -> do - (an'',r') <- markAnnListA' an' $ \a -> do - r <- markAnnotatedWithLayout stmts - return (a, r) + (an'',r') <- markAnnListA' an' $ markAnnotatedWithLayout stmts return (Right an'',r') return ((an0, l), stmts') else return ((an, l), stmts) @@ -3411,9 +3405,7 @@ instance (Typeable body, = MG (origin,an) (L (setAnchorEpa l anc ts cs) matches) exact (MG (origin,an) (L l matches)) = do - (an0,matches') <- markAnnListA' an $ \a -> do - m' <- markAnnotated matches - return (a,m') + (an0,matches') <- markAnnListA' an $ markAnnotated matches return (MG (origin, an0) (L l matches')) -- --------------------------------------------------------------------- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/eac8649f07a945988f53425b131ddd… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/eac8649f07a945988f53425b131ddd… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0

[Git][ghc/ghc][wip/T27994] 2 commits: Make niFixSubst idempotent
by Simon Peyton Jones (＠simonpj) 29 Oct '26

29 Oct '26

Simon Peyton Jones pushed to branch wip/T27994 at Glasgow Haskell Compiler / GHC Commits: ed2fd53d by Simon Peyton Jones at 2026-10-29T00:07:13+00:00 Make niFixSubst idempotent I discovered (when fixing #27994) that `niFixSubst` didn't actually generate an idempotent subsitution despite its claims. It could return a substitution that mentions domain variables on in its range, when those variables are hidden inside the kind of a type variable occurrence. As a result it can generate ill-kinded types. Boo! The fix is simple, and uses deep substitution. See Note [Finding the substitution fixpoint] I don't think it actually /caused/ any bugs, so there is no regression test. - - - - - ecc5ab3d by Simon Peyton Jones at 2026-10-29T00:07:13+00:00 Fix a nasty bug in fundeps This MR fixes #27994, which turned out to be very tricky. See Note [Exploiting closed type famililes], especially (CF7). I was ending up with TyVars (not TcTyVars) in the solver, because the substitution computed in `getRelevantBranches` mixed the two. The code was just too clever. I ended up simplifying it quite a bit, eliminating fancy footwork in a function `trim_qtvs`. I left behind Historical Note [trim_qtvs] to explain. This patch also fixes one other buglet in the same code. * Over-saturated type families were being mis-handled. See the new use of `saturatedFamApp_maybe`. - - - - - 11 changed files: - compiler/GHC/Core/Coercion/Axiom.hs - compiler/GHC/Core/Predicate.hs - compiler/GHC/Core/TyCo/FVs.hs - compiler/GHC/Core/TyCo/Subst.hs - compiler/GHC/Core/Type.hs - compiler/GHC/Core/Unify.hs - compiler/GHC/Tc/Instance/Family.hs - compiler/GHC/Tc/Solver/FunDeps.hs - + testsuite/tests/typecheck/should_compile/T27994.hs - + testsuite/tests/typecheck/should_compile/T27994a.hs - testsuite/tests/typecheck/should_compile/all.T Changes: ===================================== compiler/GHC/Core/Coercion/Axiom.hs ===================================== @@ -277,8 +277,8 @@ data CoAxBranch -- See Note [CoAxBranch type variables] , cab_eta_tvs :: [TyVar] -- ^ Eta-reduced tyvars - -- cab_tvs and cab_lhs may be eta-reduced; see - -- Note [Eta reduction for data families] + -- For a data family (only) cab_tvs and cab_lhs may be eta-reduced + -- See Note [Eta reduction for data families] , cab_cvs :: [CoVar] -- ^ Bound coercion variables -- Always empty, for now. @@ -286,8 +286,14 @@ data CoAxBranch -- in GHC.Tc.TyCl , cab_roles :: [Role] -- ^ See Note [CoAxBranch roles] + , cab_lhs :: [Type] -- ^ Type patterns to match against + -- For a data family `cab_lhs` may be shorter than the arity + -- of the data type + -- For a type family `cab_lhs` is exactly the family arity + -- See Note [Eta reduction for data families] + , cab_rhs :: Type -- ^ Right-hand side of the equality -- See Note [CoAxioms are homogeneous] ===================================== compiler/GHC/Core/Predicate.hs ===================================== @@ -36,7 +36,7 @@ module GHC.Core.Predicate ( -- Equality left-hand sides CanEqLHS(..), canEqLHS_maybe, canTyFamEqLHS_maybe, - canEqLHSKind, canEqLHSType, eqCanEqLHS + canEqLHSKind, canEqLHSType, eqCanEqLHS, saturatedFamApp_maybe ) where @@ -61,6 +61,8 @@ import GHC.Utils.Misc import GHC.Utils.Panic import GHC.Data.FastString +import GHC.Exts( inline ) + {- ********************************************************************* * * @@ -700,7 +702,9 @@ types/kinds are fully settled and zonked. -- See also Note [Ordering of implicit variables] in "GHC.Rename.HsType" scopedSort :: [Var] -> [Var] -scopedSort = go [] [] +scopedSort xs + | null xs = [] + | otherwise = go [] [] xs where go :: [Var] -- already sorted, in reverse order -> [TyCoVarSet] -- each set contains all the variables which must be placed @@ -781,13 +785,19 @@ canEqLHS_maybe xi | otherwise = canTyFamEqLHS_maybe xi -canTyFamEqLHS_maybe :: Type -> Maybe CanEqLHS -canTyFamEqLHS_maybe xi +saturatedFamApp_maybe :: Type -> Maybe (TyCon,[Type]) +saturatedFamApp_maybe xi | Just (tc, args) <- tcSplitTyConApp_maybe xi , isTypeFamilyTyCon tc , args `lengthIs` tyConArity tc - = Just $ TyFamLHS tc args + = Just (tc,args) + | otherwise + = Nothing +canTyFamEqLHS_maybe :: Type -> Maybe CanEqLHS +canTyFamEqLHS_maybe xi + | Just (tc, args) <- inline saturatedFamApp_maybe xi + = Just $ TyFamLHS tc args | otherwise = Nothing ===================================== compiler/GHC/Core/TyCo/FVs.hs ===================================== @@ -1159,10 +1159,12 @@ nice tight loops. {-# INLINE afvFolder #-} -- See Note [Any-free-var folder] afvFolder :: TyCoFolder (FV (BoundVars, TyCoVar -> Bool) DM.Any) -- 'afvFolder' is short for "any-free-var folder", good for checking --- if any /deep/ free var of a type satisfies a predicate `check_fv` +-- if any /shallow/ free var of a type satisfies a predicate `check_fv` -- The "env" of the (FV env Any) is a pair (bvs,pred) of -- the bound variables (which are definitely not free), and -- an "interesting var" predicate which the client supplies +-- +-- You can get "deep" by supplying a predicate that also looks in the kind afvFolder = TyCoFolder { tcf_view = noView -- See Note [Free vars and synonyms] , tcf_tyvar = do_tcv , tcf_covar = do_tcv @@ -1173,8 +1175,7 @@ afvFolder = TyCoFolder { tcf_view = noView -- See Note [Free vars and synonyms] -- Check the variable itself with check_fv, -- and check its kind (this is a deep folder) do_tcv tcv = MkFV $ \ (bvs,check_fv) -> - Any (not (tcv `elemVarSet` bvs) && - (check_fv tcv || runAny check_fv (afv_type (varType tcv)))) + Any (not (tcv `elemVarSet` bvs) && check_fv tcv) do_hole hole = do_tcv (coHoleCoVar hole) afv_type :: Type -> FV (BoundVars, TyVar -> Bool) DM.Any @@ -1186,16 +1187,32 @@ runAny :: (TyVar -> Bool) -> FV (BoundVars, TyCoVar -> Bool) DM.Any -> Bool {-# INLINE runAny #-} runAny check_fv fvs = DM.getAny (runFV fvs (emptyVarSet, check_fv)) +--------------------------------------------- +-- anyShallowFreeVarsOfType and friends look only for shallow free vars +-- (Currently not needed, left here in case we need it one day) +-- anyShallowFreeVarsOfType :: (TyCoVar -> Bool) -> Type -> Bool +-- {-# INLINE anyShallowFreeVarsOfType #-} +-- anyShallowFreeVarsOfType check_fv ty = runAny check_fv (afv_type ty) + +--------------------------------------------- +-- anyFreeVarsOfType and friends look for deep free vars + +deep_check :: (TyCoVar -> Bool) -> TyCoVar -> Bool +-- This is inefficient because if one tyvar occurs many times +-- in a type, it'll look in its kind at each occurrence. Boo. +deep_check check_fv tcv = check_fv tcv || anyFreeVarsOfType check_fv (varType tcv) + anyFreeVarsOfType :: (TyCoVar -> Bool) -> Type -> Bool {-# INLINE anyFreeVarsOfType #-} -anyFreeVarsOfType check_fv ty = runAny check_fv (afv_type ty) +-- Looks deeply, into the kinds of the free vars +anyFreeVarsOfType check_fv ty = runAny (deep_check check_fv) (afv_type ty) anyFreeVarsOfTypes :: (TyCoVar -> Bool) -> [Type] -> Bool {-# INLINE anyFreeVarsOfTypes #-} -anyFreeVarsOfTypes check_fv tys = runAny check_fv (afv_types tys) +anyFreeVarsOfTypes check_fv tys = runAny (deep_check check_fv) (afv_types tys) anyFreeVarsOfCo :: (TyCoVar -> Bool) -> Coercion -> Bool -anyFreeVarsOfCo check_fv co = runAny check_fv (afv_co co) +anyFreeVarsOfCo check_fv co = runAny (deep_check check_fv) (afv_co co) noFreeVarsOfType :: Type -> Bool noFreeVarsOfType ty = not $ runAny (const True) (afv_type ty) @@ -1206,6 +1223,7 @@ noFreeVarsOfTypes tys = not $ runAny (const True) (afv_types tys) noFreeVarsOfCo :: Coercion -> Bool noFreeVarsOfCo co = not $ runAny (const True) (afv_co co) + {- ************************************************************************ * * ===================================== compiler/GHC/Core/TyCo/Subst.hs ===================================== @@ -48,6 +48,8 @@ module GHC.Core.TyCo.Subst substTyCoBndr, substVarBndrUsing, checkValidSubst, isValidTCvSubst, + + deepSubstTy ) where import GHC.Prelude @@ -775,9 +777,15 @@ substThetaUnchecked = substTysUnchecked -- building no thunks. See Note [Specialising mappers] in GHC.Core.TyCo.Make type SubstM = StrictReaderT Subst StrictIdentity -runSubstM :: SubstM a -> Subst -> a -runSubstM m subst = runStrictIdentity (runStrictReaderT m subst) +runSubstM :: forall a. (a -> SubstM a) -> Subst -> a -> a {-# INLINE runSubstM #-} +runSubstM tx subst x = runStrictIdentity (runStrictReaderT (tx x) subst) + +subst_ty :: Subst -> Type -> Type +subst_co :: Subst -> Coercion -> Coercion +(subst_ty, subst_co) + = case mapTyCo substTyCoMapper of + (ms_ty, _, ms_co, _) -> (runSubstM ms_ty, runSubstM ms_co) substTyCoMapper :: TyCoMapper SubstM substTyCoMapper @@ -785,8 +793,8 @@ substTyCoMapper , tcm_covar = subst_cv , tcm_hole = subst_ch , tcm_tycobinder = tcv_bndr - , tcm_tcapp_ty = tcapp_ty - , tcm_tcapp_co = tcapp_co + , tcm_tcapp_ty = nopTcAppTy + , tcm_tcapp_co = nopTcAppCo } where subst_tv tv = StrictReaderT $ \subst -> return (substTyVar subst tv) @@ -802,23 +810,15 @@ substTyCoMapper -- Avoid allocation in this very -- common case (E.g. Int, LiftedRep etc) - tcapp_ty :: Type -> TyCon -> [Type] -> SubstM Type - tcapp_ty ty tc tys' - | null tys' = return ty - | otherwise = return (mkTyConApp tc tys') - tcapp_co :: Coercion -> Role -> TyCon -> [Coercion] -> SubstM Coercion - tcapp_co co r tc cos' - | null cos' = return co - | otherwise = return (mkTyConAppCo r tc cos') +nopTcAppTy :: Type -> TyCon -> [Type] -> SubstM Type +nopTcAppTy ty tc tys' + | null tys' = return ty + | otherwise = return (mkTyConApp tc tys') -subst_ty :: Subst -> Type -> Type -subst_co :: Subst -> Coercion -> Coercion -(subst_ty, subst_co) - = case mapTyCo substTyCoMapper of - (ms_ty, _, ms_co, _) -> (run ms_ty, run ms_co) - where - run :: forall a. (a -> SubstM a) -> Subst -> a -> a - run ms subst x = runSubstM (ms x) subst +nopTcAppCo :: Coercion -> Role -> TyCon -> [Coercion] -> SubstM Coercion +nopTcAppCo co r tc cos' + | null cos' = return co + | otherwise = return (mkTyConAppCo r tc cos') substTyVar :: Subst -> TyVar -> Type substTyVar (Subst _ _ tenv _) tv @@ -1019,3 +1019,75 @@ it is /very/ valuable to /keep/ it empty: If the substitution is empty, this is a no-op -- but only if it is empty! -} + + +{- ********************************************************************** +%* * + Deep substitution +%* * +%********************************************************************* -} + +{- Note [Deep substitution] +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Deep substitution applies the substitution to the /kinds/ of un-substituted +type variables. Suppose the substitution is + [ k :-> Type ] +and we apply it to the type + Proxy k (a :: k) +Ordinary shallow substitution will give + Proxy Type (a :: k) +which is ill-kinded. Deep substitution will give + Proxy Type (a :: Type) + +Ordinary (shallow) substitution is usually fine, because we will substitute for +`a` as well, but not always. In particular, the pure unifier in in GHC.Core.Unify +uses `niFixSubst` to find a fixpoint, and we really don't want domain tyvars +appearing in the range.See Note [Finding the substitution fixpoint]. + +Deep substitution risks being rather inefficient because it re-applies the +subsitution to the kind at /each occurrence/ of a free variable. We could +perhaps find the free vars, and build a re-kinding substitution, but that's a +bit more complicated. +-} + +deepSubstTyCoMapper :: TyCoMapper SubstM +deepSubstTyCoMapper + = TyCoMapper { tcm_tyvar = subst_tv + , tcm_covar = subst_cv + , tcm_hole = subst_ch + , tcm_tycobinder = tcv_bndr + , tcm_tcapp_ty = nopTcAppTy + , tcm_tcapp_co = nopTcAppCo + } + where + subst_tv tv = StrictReaderT $ \subst -> return (deepSubstTyVar subst tv) + subst_cv cv = StrictReaderT $ \subst -> return (deepSubstCoVar subst cv) + subst_ch ch = StrictReaderT $ \subst -> return (deepSubstCoHole subst ch) + + tcv_bndr :: TyCoVar -> ForAllTyFlag -> (TyCoVar -> SubstM r) -> SubstM r + tcv_bndr tcv _vis k + = StrictReaderT $ \subst -> + let !(subst', tcv') = substVarBndrUsing deepSubstTy subst tcv + in runStrictReaderT (k tcv') subst' + +deepSubstTy :: Subst -> Type -> Type +deepSubstTy = case mapTyCo deepSubstTyCoMapper of + (subst_ty, _, _, _) -> runSubstM subst_ty + +deepSubstTyVar :: Subst -> TyVar -> Type +deepSubstTyVar subst@(Subst _ _ tenv _) tv + = assert (isTyVar tv) $ + case lookupVarEnv tenv tv of + Just ty -> ty + Nothing -> TyVarTy (updateVarType (deepSubstTy subst) tv) + +deepSubstCoVar :: Subst -> CoVar -> Coercion +deepSubstCoVar subst@(Subst _ _ _ cenv) cv + = case lookupVarEnv cenv cv of + Just co -> co + Nothing -> CoVarCo (updateVarType (deepSubstTy subst) cv) + +deepSubstCoHole :: Subst -> CoercionHole -> Coercion +-- See Note [Substituting in a coercion hole] +deepSubstCoHole subst h@(CH { ch_co_var = cv }) + = HoleCo (h { ch_co_var = updateVarType (deepSubstTy subst) cv }) ===================================== compiler/GHC/Core/Type.hs ===================================== @@ -483,7 +483,7 @@ expandTypeSynonyms ty expandTypeSynonymsX :: Subst -> Type -> Type expandTypeSynonymsX = case mapTyCo expandTypeSynonymMapper of - (exp_ty, _, _, _) -> \subst ty -> runSubstM (exp_ty ty) subst + (exp_ty, _, _, _) -> runSubstM exp_ty expandTypeSynonymMapper :: TyCoMapper SubstM -- Just like substitution, but treat TyConApp specially ===================================== compiler/GHC/Core/Unify.hs ===================================== @@ -28,13 +28,12 @@ import GHC.Types.Var.Env import GHC.Types.Var.Set import GHC.Core.Type hiding ( getTvSubstEnv ) import GHC.Core.Coercion hiding ( getCvSubstEnv ) -import GHC.Core.Predicate( scopedSort ) import GHC.Core.TyCon import GHC.Core.Predicate( CanEqLHS(..), canEqLHS_maybe ) import GHC.Core.TyCon.Env import GHC.Core.TyCo.Rep import GHC.Core.TyCo.Compare ( eqType, tcEqType, tcEqTyConAppArgs ) -import GHC.Core.TyCo.Subst ( mkTvSubst ) +import GHC.Core.TyCo.Subst ( mkTvSubst, deepSubstTy ) import GHC.Core.Map.Type import GHC.Core.Multiplicity @@ -1063,21 +1062,31 @@ During unification we use a TvSubstEnv/CvSubstEnv pair that is Note [Finding the substitution fixpoint] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Finding the fixpoint of a non-idempotent substitution arising from a -unification is much trickier than it looks, because of kinds. Consider +We want an /idempotent/ substitution, so `niFixSubst` takes a non-idempotent +one and composes it with itself until it becomes idempotent. Thus if the original +is S0, we compute S1 = S0 o S0, S2 = S1 o S1, etc. + +A variable chain v0 :-> v1 :-> ... :-> vm therefore takes about log2 m rounds, +The unifier can build such. For example: + (a, a, a, ..., a) ~ (x1, x2, x3, ..., xm) +produces a :-> x1, x1 :-> x2, ..., x(m-1) :-> xm. + +Finding the fixpoint is much trickier than it looks, because of kinds. +Consider T k (H k (f:k)) ~ T * (g:*) If we unify, we get the substitution [ k -> * , g -> H k (f:k) ] +This isn't idempotent: it mentions `k` in the range. + To make it idempotent we don't want to get just [ k -> * , g -> H * (f:k) ] -We also want to substitute inside f's kind, to get - [ k -> * - , g -> H k (f:*) ] -If we don't do this, we may apply the substitution to something, -and get an ill-formed type, i.e. one where typeKind will fail. +because (H * (f:k)) is itself ill-kinded, and `typeKind` will fail. This happened, for example, in #9106. +So (yuckily) we must also substitute inside f's kind, to get + [ k -> * + , g -> H * (f:*) ] It gets worse. In #14164 we wanted to take the fixpoint of this substitution @@ -1090,84 +1099,45 @@ the invocation of F! We don't have a function that recursively applies substitutions inside the kinds of variable occurrences (and probably rightly so). -So, we work as follows: - - 1. Start with the current substitution (which we are - trying to fixpoint - [ xs :-> F a (z :: a) (rest :: G a (z :: a)) - , a :-> b ] - - 2. Take all the free vars of the range of the substitution: - {a, z, rest, b} - NB: the free variable finder closes over - the kinds of variable occurrences - - 3. If none are in the domain of the substitution, stop. - We have found a fixpoint. - - 4. Remove the variables that are bound by the substitution, leaving - {z, rest, b} - - 5. Do a topo-sort to put them in dependency order: - [ b :: *, z :: a, rest :: G a z ] - - 6. Apply the substitution left-to-right to the kinds of these - tyvars, extending it each time with a new binding, so we - finish up with - [ xs :-> ..as before.. - , a :-> b - , b :-> b :: * - , z :-> z :: b - , rest :-> rest :: G b (z :: b) ] - Note that rest now has the right kind - - 7. Apply this extended substitution (once) to the range of the - /original/ substitution. (Note that the extended substitution - would go on forever if you tried to find its fixpoint, because it - maps z to z.) - - 8. And go back to step 1 - -In Step 6 we use the free vars from Step 2 as the initial -in-scope set, because all of those variables appear in the -range of the substitution, so they must all be in the in-scope -set. But NB that the type substitution engine does not look up -variables in the in-scope set; it is used only to ensure no -shadowing. +Ugh! This whole thing is pretty horrible. But the solution is simple enough: + + Use deep substitution (deepSubstTy) + +(deepSubst `s` (Proxy k (a::k)) applies the substitution `s` not only to +the occurrence of `k` in the argument of `Proxy`, but also to the `k` inside +the kind of `a`. See Note [Deep substitution] in GHC.Core.TyCo.Subst. + +So `niFixSubst` just applies the substitution /deeply/ to the types in its +range, until we reach a fixed point. Easy! -} niFixSubst :: InScopeSet -> TvSubstEnv -> Subst -- Find the idempotent fixed point of the non-idempotent substitution -- This is surprisingly tricky: -- see Note [Finding the substitution fixpoint] --- ToDo: use laziness instead of iteration? niFixSubst in_scope tenv - | not_fixpoint = niFixSubst in_scope (mapVarEnv (substTy subst) tenv) - | otherwise = tenv_subst + | isEmptyVarEnv tenv -- Fast path for common case + = mkEmptySubst in_scope + | otherwise + = -- pprTrace "niFixSubst" (vcat [ text "In: " <+> ppr tenv + -- , text "Out:" <+> ppr (S.getTvSubstEnv subst) ]) + subst where - tenv_subst = mkTvSubst in_scope tenv -- This is our starting point - - range_tvs :: [TyVar] - range_tvs = nonDetVarSetElems $ - tyCoVarsOfTypes $ - nonDetEltsUFM tenv - -- It's OK to use nonDetEltsUFM (twice) here because - -- the order of range_tvs is immaterial - - not_fixpoint = any in_domain range_tvs - in_domain tv = tv `elemVarEnv` tenv - - free_tvs = scopedSort (filterOut in_domain range_tvs) + subst = niFixSubst' 1 in_scope tenv - -- See Note [Finding the substitution fixpoint], Step 6 - subst = foldl' add_free_tv tenv_subst free_tvs - - add_free_tv :: Subst -> TyVar -> Subst - add_free_tv subst tv - | isTyVar tv = extendTvSubst subst tv (mkTyVarTy tv') - | otherwise = subst -- Ignore free coercion variables - where - tv' = updateTyVarKind (substTy subst) tv +niFixSubst' :: Int -> InScopeSet -> TvSubstEnv -> Subst +niFixSubst' n in_scope tenv_orig + | fixpoint_found = mkTvSubst in_scope tenv_orig + | otherwise = niFixSubst' (n+1) in_scope tenv_new + where + in_dom tv = tv `elemVarEnv` tenv_orig + tenv_new = mapVarEnv (deepSubstTy subst) tenv_orig + subst = mkTvSubst in_scope tenv_orig + + fixpoint_found = nonDetStrictFoldUFM and_fix True tenv_orig + and_fix :: Type -> Bool -> Bool + and_fix _ False = False + and_fix ty _ = not (anyFreeVarsOfType in_dom ty) {- ************************************************************************ ===================================== compiler/GHC/Tc/Instance/Family.hs ===================================== @@ -16,6 +16,7 @@ import GHC.Prelude import GHC.Driver.DynFlags import GHC.Driver.Env +import GHC.Core.Predicate( saturatedFamApp_maybe ) import GHC.Core.FamInstEnv import GHC.Core.Coercion import GHC.Core.TyCon @@ -715,15 +716,9 @@ reportInjectivityErrors dflags fi_ax axiom inj when wrong_bare_rhs $ reportBareVariableInRHSErr fam_tc bare_variables axiom -- | Is type headed by a type family application? -isTFHeaded :: Type -> Bool -- See Note [Verifying injectivity annotation], case 3. -isTFHeaded ty | Just ty' <- coreView ty - = isTFHeaded ty' -isTFHeaded ty | (TyConApp tc args) <- ty - , isTypeFamilyTyCon tc - = args `lengthIs` tyConArity tc -isTFHeaded _ = False - +isTFHeaded :: Type -> Bool +isTFHeaded ty = isJust (saturatedFamApp_maybe ty) -- | If a RHS is a bare type variable return a set of LHS patterns that are not -- bare type variables. ===================================== compiler/GHC/Tc/Solver/FunDeps.hs ===================================== @@ -24,16 +24,16 @@ import GHC.Tc.Types.Constraint import GHC.Core.FamInstEnv import GHC.Core.Coercion import GHC.Core.TyCo.Rep( Type(..) ) -import GHC.Core.Predicate( EqRel(..) ) +import GHC.Core.TyCo.Subst( deepSubstTy ) +import GHC.Core.Predicate( EqRel(..), saturatedFamApp_maybe ) import GHC.Core.TyCon import GHC.Core.Unify( AmIUnifying(..) , tcUnifyTysForInjectivity, typeListsAreApart ) import GHC.Core.Coercion.Axiom -import GHC.Core.TyCo.Subst( elemSubst, extendTvSubstWithClone ) import GHC.Builtin.WiredIn.TypeLits( tryInteractTopFam, tryInteractInertFam ) -import GHC.Types.Var( TyVar, tyVarKind, setTyVarKind ) +import GHC.Types.Var( tyVarKind ) import GHC.Types.Var.Set import GHC.Types.Var.Env( mkInScopeSet ) @@ -57,7 +57,6 @@ Here is our plan for dealing with functional dependencies * send the current constraint back to the start of the pipeline; it might now be soluble, and it probably isn't inert - * Once a unification is done, it can't be un-done. So we must be very careful not to threaten completeness by doing a unification when there might be another way to solve the constraint. @@ -613,6 +612,7 @@ tryFamEqFunDeps mode eqs_for_me fam_tc work_args mkTopFamEqFDs :: TyCon -> [TcType] -> EqCt -> SolverStage [FunDepEqns] mkTopFamEqFDs fam_tc work_args work_item + -- NB: work_args exactly saturate fam_tc | isOpenTypeFamilyTyCon fam_tc , Injective inj_flags <- tyConInjectivityInfo fam_tc = -- Open, injective type families @@ -709,24 +709,24 @@ getRelevantBranches ax work_args work_rhs , cab_lhs = lhs_tys , cab_rhs = rhs_ty }) | Just subst <- tcUnifyTysForInjectivity Unifying (rhs_ty:lhs_tys) work_tys - -- See (RW1)-(RW3) in Note [Relevance] - -- This is a pre-/unification/ check (not matching) - -- so `subst` can bind both `qtvs` and fvs(work_tys) - , let (subst',qtvs') = trim_qtvs subst qtvs - lhs_tys' = substTys subst' lhs_tys - rhs_ty' = substTy subst' rhs_ty - branch_tys' = rhs_ty' : lhs_tys' + -- See (RW1)-(RW3) in Note [Relevance] + , let lhs_tys' = map (deepSubstTy subst) lhs_tys + -- deepSubstTy: see (CF8) in Note [Exploiting closed type families] , noEarlierMatchCheck lhs_tys' branch -- See (RW4) in Note [Relevance] - = Just (FDEqns { fd_qtvs = qtvs' - , fd_eqs = zipWith Pair branch_tys' work_tys }) - -- Do not apply the substition to `work_tys`; - -- See (CF7) in Note [Exploiting closed type families] + = -- Ignore the substitution, except for `noEarlierMatchCheck` + -- See (CF7) in Note [Exploiting closed type families] + Just (FDEqns { fd_qtvs = qtvs + , fd_eqs = zipWith Pair (rhs_ty:lhs_tys) work_tys }) + -- Do not apply the substitution to `work_tys`; + -- See (CF9) in Note [Exploiting closed type families] + | otherwise = Nothing mkTopOpenFamEqFDs :: TyCon -> [Bool] -> [TcType] -> EqCt -> TcS [FunDepEqns] -- Implements (INJFAM:Wanted/top-open) -- See Note [Exploiting open type families] +-- NB: work_args exactly saturate fam_tc mkTopOpenFamEqFDs fam_tc inj_flags work_args (EqCt { eq_rhs = work_rhs }) = do { fam_envs <- getFamInstEnvs ; let branches :: [CoAxBranch] @@ -738,12 +738,11 @@ mkTopOpenFamEqFDs fam_tc inj_flags work_args (EqCt { eq_rhs = work_rhs }) do_one (CoAxBranch { cab_tvs = branch_qtvs , cab_lhs = branch_lhs_tys , cab_rhs = branch_rhs }) - | Just subst <- tcUnifyTysForInjectivity Matching [branch_rhs] [work_rhs] - , let (subst', qtvs') = trim_qtvs subst branch_qtvs - branch_lhs_tys' = substTys subst' branch_lhs_tys - -- No need for an apartness check here; for open type families - -- there are no "preceding" equations - = Just (mkInjectivityFDEqn inj_flags qtvs' branch_lhs_tys' work_args) + | Just {} <- tcUnifyTysForInjectivity Matching [branch_rhs] [work_rhs] + -- No need for an apartness check here; + -- for open type families there are no "preceding" equations + -- Both branch_lhs_tys and work_args exactly saturate fam_tc + = Just (mkInjectivityFDEqn branch_qtvs inj_flags branch_lhs_tys work_args) | otherwise = Nothing @@ -756,9 +755,9 @@ mkLocalFamEqFDs eqs_for_me fam_tc inj_flags work_args work_rhs = do { let -- eqns_from_inerts: see (INJFAM:Wanted/other) eqns_from_inerts = mapMaybe do_one eqs_for_me -- eqns_from_self: see (INJFAM:Wanted/Self) - eqns_from_self = case tcSplitTyConApp_maybe work_rhs of - Just (tc,rhs_tys) | tc==fam_tc -> [mk_eqn rhs_tys] - _ -> [] + eqns_from_self = case saturatedFamApp_maybe work_rhs of + Just (tc, rhs_tys) | fam_tc == tc -> [mk_eqn rhs_tys] + _ -> [] ; return (eqns_from_inerts ++ eqns_from_self) } where @@ -768,7 +767,10 @@ mkLocalFamEqFDs eqs_for_me fam_tc inj_flags work_args work_rhs | otherwise = Nothing do_one _ = pprPanic "interactFunEq 2" (ppr fam_tc) -- TyVarLHS - mk_eqn iargs = mkInjectivityFDEqn inj_flags [] work_args iargs + mk_eqn iargs = mkInjectivityFDEqn [] inj_flags work_args iargs + + +{- See Historical Note [trim_qtvs] trim_qtvs :: Subst -> [TyVar] -> (Subst, [TyVar]) -- Tricky stuff: see (OF1) in Note [Exploiting open type families] @@ -790,7 +792,7 @@ trim_qtvs unif_subst qtvs subst' = extendTvSubstWithClone subst tv tv' -- extendTvSubstWithClone: see (OF1b) in -- Note [Exploiting open type families] - +-} ----------------------------------------- -- Built-in type families @@ -883,15 +885,22 @@ mkLocalBuiltinFamEqFDs eqs_for_me fam_tc ops work_args work_rhs ; return (eqns_from_inerts ++ eqns_from_self) } -------------------- -mkInjectivityFDEqn :: [Bool] -- Injectivity flags - -> [TcTyVar] -- Quantify these +mkInjectivityFDEqn :: [TcTyVar] -- Quantify these + -> [Bool] -- Injectivity flags -> [TcType] -> [TcType] -- Make these equal -> FunDepEqns -- When F s1 s2 s3 ~ F t1 t2 t3, and F has injectivity info [True,False,True] -- return the FDEqns { fd_eqs = [Pair s1 t1, Pair s3 t3] } --- The injectivity flags [Bool] will not all be False, but nothing goes wrong if they are -mkInjectivityFDEqn inj_args qtvs lhs_args rhs_args - = FDEqns { fd_qtvs = qtvs, fd_eqs = eqs } +-- The injectivity flags [Bool] will not all be False, +-- but nothing goes wrong if they are +mkInjectivityFDEqn qtvs inj_args lhs_args rhs_args + = assertPpr (let len = length inj_args in + len == length lhs_args && len == length rhs_args) + (vcat [ text "inj_args:" <+> ppr inj_args + , text "lhs_args:" <+> ppr lhs_args + , text "rhs_args:" <+> ppr rhs_args ]) $ + -- Need inj_args lhs_args and rhs_args to be same length for zip3 + FDEqns { fd_qtvs = qtvs, fd_eqs = eqs } where eqs = [ Pair lhs_arg rhs_arg | (True, lhs_arg, rhs_arg) <- zip3 inj_args lhs_args rhs_args ] @@ -1104,7 +1113,7 @@ the Injective Type Families paper. For open type families, suppose we have [W] F alpha ~ Int->Int -Then, since F has been checked as injective, and the RHS `a->a` (pre-)unifies +Then, since F has been checked as injective, and the RHS `a->a` (pre-)matches with the `Int->Int` in the Wanted with substitution [a :-> Int], we can add a derived FunDepEqns FDEqns{ fd_qtvs = [], fd_eqs = [ [Int] ~ alpha ] } @@ -1119,7 +1128,7 @@ instance are bound by the pre-unification step. Suppose we have [W] F alpha ~ [ty] -Then, the RHS [G a] pre-unifies with the RHS of the Wanted, but with an +Then, the RHS [G a] pre-matches with the RHS of the Wanted, but with an empty substitution, which does not fix `a`. So we get this FunDepEqns: FDEqns{ fd_qtvs = a, fd_eqs = [ Maybe a ~ alpha ] } (note the fd_qtvs=a) and thereby emit (via `instantiateFunDepEqns`) @@ -1133,36 +1142,44 @@ In general we have: type instance forall qs. F t1 ... tn = rhs [W] F a1 ... an ~ b -Suppose `b` pre-unifies with `rhs`, with substitution `S` that binds -some (but perhaps not all) of the `qs`. Lets say - unbound_qs = qs \ dom(S) - -Then we generate the FunDepEqs - FDEqns{ fd_qtvs = unbound_qs - , fd_eqs = [ S(t1) ~ S(a1), S(t3) ~ S(a3) ] - -These 'fd_qtvs' are subsequently instantiated to fresh metavariables by -'instantiateFunDepEqns'. - -(OF1) The computation of `unbound_qs` is done by `trim_qtvs`, but we - have to be careful. Suppose (OF1a) - qs = [ k::*, a::k ] - S = [ k :-> UnliftedType ] - Then we want - unbound_qs = [ a::UnliftedType ] -- Must apply S to q's kind!! - - And (OF1b), although I'm not sure if this can acutally happen: - qs = [ k::*, a::(k::*), b::(a::k) ] - S = [ k :-> UnliftedType ] - We want - unbound_qs = [ a::UnliftedType, b::(a::UnliftedType) ] - Note that the occurrence of `a` in b's kind should be replaced by the - newly-kinded `a` not the original `a::k`. - - This fancy footwork for `fd_qtvs` is done by `trim_qtvs`. It - * Drops a quantified tyvar that /is/ matched - * Substitutes in the kind of a quantified tyvar `a` that is /not/ matched - * Adds the substitution [a::k :-> a::S(k)] to the substitution +Suppose `b` pre-matches with `rhs`, with substitution `S`. We /ignore/ +`S` and simply generate the FunDepEqs + FDEqns{ fd_qtvs = qs, fd_eqs = [ t1 ~ a1, t3 ~ a3 ] +These 'fd_qtvs' are subsequently instantiated to fresh meta-variables by +'instantiateFunDepEqns'. See Historical Note [trim_qtvs]. + +Historical Note [trim_qtvs] +~~~~~~~~~~~~~~~~~~~~~~~~~~~ +In Note [Exploiting open type families], we previously tried to make use of +the matching substitution `S`, by defining + unbound_qs = qs \ dom(S) +so that we could make fresh meta-variables for only the unbound `qs`. +But it turned out to be tricky. Suppose (TQ1) + qs = [ k::*, a::k ] + S = [ k :-> UnliftedType ] +Then we want + unbound_qs = [ a::UnliftedType ] -- Must apply S to q's kind!! + +And (TQ2), although I'm not sure if this can actually happen: + qs = [ k::*, a::(k::*), b::(a::k) ] + S = [ k :-> UnliftedType ] +We want + unbound_qs = [ a::UnliftedType, b::(a::UnliftedType) ] +Note that the occurrence of `a` in b's kind should be replaced by the +newly-kinded `a` not the original `a::k`. + +This fancy footwork for `fd_qtvs` used to be done by `trim_qtvs`. It + * Drops a quantified tyvar that /is/ matched + * Substitutes in the kind of a quantified tyvar `a` that is /not/ matched + * Adds the substitution [a::k :-> a::S(k)] to the substitution + +(TQ3) This was hard enough for open type type families, which use matching. +But it was ridiculously hard for /closed/ type families which use pre-/unification/ +so that the substitution `S` has both meta-tyvars and instance-head tyvars in +the range. + +It's much easier to drop all this and just make fresh meta-tyvars for +all the quantified type variables. Note [Exploiting closed type families] see `mkTopClosedFamEqFDs` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1194,9 +1211,6 @@ More precisely, here is the very clever Closed Family Fundep Algorithm (CFFA) with fresh unification vars in lhs and rhs for the quantified variables of the equation. - Actually we take advantage of the substitution generated by the relevance - check, see (CF7) below. - See Note [Relevance] for what "relevant" means. We need to take care about non-termination; see (CF3). @@ -1391,26 +1405,59 @@ Key point: equations that are not relevant do not need to be considered for fund (CF7) Relevance. When computing relevance in the Closed Family Fundep Algorithm, for [W] F wlhs1 wlhs2 ~ wrhs, against an equation F lhs1 lhs2 ~ rhs we get back a pre-unifying substitution S that makes the two as equal as poss. - Then we use `trim_qtvs` in exactly the same way as in - (OF1) of Note [Exploiting open type families], to generate qtvs' and S' - Then we generate - FDEqns { fd_qtvs = qtvs', fd_eqs [ S'(lhs1) ~ wlhs1 - , S'(lhs2) ~ wlhs2 - , S'(rhs) ~ wrhs ] } - Notice that we do /not/ apply S' to wlhs/wrhs. These type contain unification - variables that we must actually unify! Example + This substitution can bind both /quantified/ type variables from the instance, + and /unification variables/ from the solver. Moreover, both kind of type variable + can appear in the range of the substitution. + + That's pretty complicated, so we are very cautious about using the returned + substitution. I had a long struggle (#27994) when we were previously trying to + apply that returned substitution as an optimisation. But because type variables + can occur in kinds, it was a losing battle -- I ended up with TyVars from the + instance head escaping into the solver. Consider + + type F :: forall k. k -> * + type instance F k a where + F (k->*) (b::k->*) = Int + + [W] F kappa (alpha::kappa) ~ Int + + We unify the Wanted with the equation to get the substitution + [ kappa :-> k->*, b :-> alpha ] + I tried using `trim_qtvs` (see Historical Note [trim_qtvs]) but my head hurt + FAR too much: #27994 is an excellent example. I gave up and now just ignore + the substitution, making a fresh meta-var for each quantified variable. + +(CF8) When computing relevance (Note [Relevance] (R2)) we use `noEarlierMatch` + on S(lhs). But we must use /deep/ substitution on LHS. Consider + type family F (a :: k) r where + F @Type (Maybe Bool) r = r -- B0 + F @Type ((f :: Type -> Type) Int) r = Bool -- B1 + F @k ((g :: Type -> k) Int) r = r -- B2 + + [W] F @Type alpha Char ~ Char + The `tcUnifyTysForInjectivity` will return substitution + [k :-> Type, r :-> Char, alpha :-> (g :: Type->Type) Int], with g unbound + If we used `substTy` on the LHS-tys for B2 we'd get + lhs_tys' = [Type, (g :: Type -> k) Int, Char] + That's ill-kinded because of the free `k`; and (worse) when `noEarlierMatch` matches + lhs_tys' against earlier equations, (B1) will fail to match. So (B2) will be + judged relevant, even though it isn't. Bad, bad. Use deep substitution. + +(CF9) Notice that, when generating the fundep equations for the solver to deal + with, we do /not/ apply the pre-unifying substitution S. The types wlhs,wrhs + contain meta-variables that we must actually unify! Example type family F a b where F [a] (Maybe b) = (a, G b) [W] F alpha beta ~ (Int, [gamma]) Pre-unification gives [alpha :-> [a], beta :-> Maybe b, a :-> Int] - This substitution eliminates the quantified variable `a` but not `b`, so we - want this: - FDEqns { fd_qtvs = [b], fd_eqs [ [Int] ~ alpha - , Maybe b ~ beta - , (Int,G b) ~ (Int, [gamma]) ] } - - Notice also that by applying the substitution to the template but not to the - Wanted, we uphold the `fd_eqs` invariant in `FDEqns` (see its data type decl) + We don't want to substitute `alpha` away! In fact we just generate these eqns: + FDEqns { fd_qtvs = [a,b], fd_eqs [ [a] ~ alpha + , Maybe b ~ beta + , (a, G b) ~ (Int, [gamma]) ] } + with `fd_qtvs` being the quantified variables of the equation, the LHSs + being the types from the equation and the RHSs being the types from the Wanted. + Notice that we thereby uphold the `fd_eqs` invariant in `FDEqns` (see its data + type decl). Note [Cache-caused loops] ~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -1517,8 +1564,8 @@ instantiateFunDepEqns (FDEqns { fd_qtvs = qtvs, fd_eqs = eqs }) rev_eqs = reverse eqs -- (reverse eqs): See Note [Reverse order of fundep equations] - -- The kinds of `qtvs` can have free variables, because of `trim_qtvs` - -- Example: T27828b + -- The kinds of `qtvs` can have free variables; + -- in the FDEqns produced by GHC.Tc.Instance.FunDeps.improveFromInstEnv empty_subst = mkEmptySubst $ mkInScopeSet $ tyCoVarsOfTypes (map tyVarKind qtvs) ===================================== testsuite/tests/typecheck/should_compile/T27994.hs ===================================== @@ -0,0 +1,48 @@ +{-# LANGUAGE AllowAmbiguousTypes #-} +{-# LANGUAGE DataKinds #-} +{-# LANGUAGE FunctionalDependencies #-} +{-# LANGUAGE GADTs #-} +{-# LANGUAGE TypeFamilies #-} +module T27994a (l2) where + +import Data.Kind (Constraint, Type) + +l1 :: [String] +l1 = ["hello", "world!"] + +l2 :: [Int] +l2 = gmap (HCons length HNil) l1 + +----- + +type HList :: [Type] -> Type +data HList p where + HNil :: HList '[] + HCons :: x -> HList xs -> HList (x:xs) + +type ($$) :: k -> [Type] -> Type +type family s $$ p where + (s :: Type) $$ '[] = s + (s :: Type -> k) $$ (a:b) = s a $$ b + +type Doms :: [Type] -> [Type] +type family Doms t where + Doms '[] = '[] + Doms ((a -> b) : t) = a : Doms t + +type Codoms :: [Type] -> [Type] +type family Codoms t where + Codoms '[] = '[] + Codoms ((a -> b) : t) = b : Codoms t + +type GMap :: k -> [Type] -> Constraint +class GMap (s :: k) t | t -> k where + gmap :: HList t -> s $$ Doms t -> s $$ Codoms t + +instance GMap [] '[a -> b] where + gmap :: HList '[a -> b] -> [a] -> [b] + gmap (HCons f HNil) = go + where + go :: [a] -> [b] + go [] = [] + go (x:xs) = f x : go xs ===================================== testsuite/tests/typecheck/should_compile/T27994a.hs ===================================== @@ -0,0 +1,53 @@ +{-# LANGUAGE DataKinds, TypeFamilies, PolyKinds, GADTs, AllowAmbiguousTypes #-} +module T27994 where + +import Data.Kind (Type) +import Data.Proxy (Proxy) + +type F :: forall{k}. k -> Type -> Type +type family F s x where + F (s :: Type -> k) (Proxy s) = Int + +f :: forall k (s :: k) x. F s x -> x +f _ = undefined + +g :: Proxy Maybe -> () +g _ = () + +data T a where + MkT :: T Int + +test :: T a -> () +test t = (\y -> case t of MkT -> g y) (f (3 :: Int)) + +{- [W] F @k_azX (s_azY::k_azX) p_azN ~ Int + +unify [*->k, s::*->k, Proxy @(*->k) (s::*->k), Int] + ~ [k_azX, s_azY::k_azX, p_azN, Int] + + --> [ k_azX :-> (* -> k) + , s :-> s_azY::k_azX + , p_azN :-> Proxy @(*->k) (s::*->k) ] + + fixpt --> [ k_azX :-> Type -> k + , s :-> s_azY::(Type -> k) + , p_azN :-> Proxy @(Type->k) (s::k_azX) ] + +Generate a [W] for each pair + [W] k ~ k_azX + [W] s ~ s_azY + [W] Proxy s ~ p_azN + [W] Int ~ Int +As an opt (claim) apply subst first + [W] k_azX ~ k_azX + [W] s_azY ~ s_azY + [W] Proxy s_azY ~ p_azN + [W] Int ~ Int + +-- [W] F @k_azX s_azY[tau] p_azN ~ Int +-- Want: fresh k' s' +-- [W] k' ~ k_azX +-- [W] s':k' ~ s_azY:k_azX +-- [W] Proxy s' ~ x +-} + ===================================== testsuite/tests/typecheck/should_compile/all.T ===================================== @@ -972,4 +972,6 @@ test('T27557', normal, compile, ['']) test('T27664', normal, compile, ['']) test('T27916', normal, compile, ['']) test('T26543_orig', normal, compile, ['']) +test('T27994', normal, compile, ['']) +test('T27994a', normal, compile, ['']) View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/6af2da8dddf42eb3aa8ae999b3a8b5… -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/6af2da8dddf42eb3aa8ae999b3a8b5… You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help

1 0