[Git][ghc/ghc][wip/bytecode-serialize-3] 16 commits: level imports: Check the level of exported identifiers

Cheng Shao pushed to branch wip/bytecode-serialize-3 at Glasgow Haskell Compiler / GHC Commits: 246b7853 by Matthew Pickering at 2025-08-07T06:58:30-04:00 level imports: Check the level of exported identifiers The level imports specification states that exported identifiers have to be at level 0. This patch adds the requird level checks that all explicitly mentioned identifiers occur at level 0. For implicit export specifications (T(..) and module B), only level 0 identifiers are selected for re-export. ghc-proposal: https://github.com/ghc-proposals/ghc-proposals/pull/705 Fixes #26090 - - - - - 358bc4fc by fendor at 2025-08-07T06:59:12-04:00 Bump GHC on darwin CI to 9.10.1 - - - - - 1903ae35 by Matthew Pickering at 2025-08-07T12:21:10+01:00 ipe: Place strings and metadata into specific .ipe section By placing the .ipe metadata into a specific section it can be stripped from the final binary if desired. ``` objcopy --remove-section .ipe <binary> upx <binary> ``` Towards #21766 - - - - - c80dd91c by Matthew Pickering at 2025-08-07T12:22:42+01:00 ipe: Place magic word at the start of entries in the .ipe section The magic word "IPE\nIPE\n" is placed at the start of .ipe sections, then if the section is stripped, we can check whether the section starts with the magic word or not to determine whether there is metadata present or not. Towards #21766 - - - - - cab42666 by Matthew Pickering at 2025-08-07T12:22:42+01:00 ipe: Use stable IDs for IPE entries IPEs have historically been indexed and reported by their address. This makes it impossible to compare profiles between runs, since the addresses may change (due to ASLR) and also makes it tricky to separate out the IPE map from the binary. This small patch adds a stable identifier for each IPE entry. The stable identifier is a single 64 bit word. The high-bits are a per-module identifier and the low bits identify which entry in each module. 1. When a node is added into the IPE buffer it is assigned a unique identifier from an incrementing global counter. 2. Each entry already has an index by it's position in the `IpeBufferListNode`. The two are combined together by the `IPE_ENTRY_KEY` macro. Info table profiling uses the stable identifier rather than the address of the info table. The benefits of this change are: * Profiles from different runs can be easily compared * The metadata can be extracted from the binary (via the eventlog for example) and then stripped from the executable. Fixes #21766 - - - - - 2860a9a5 by Simon Peyton Jones at 2025-08-07T20:29:18-04:00 In TcSShortCut, typechecker plugins should get empty Givens Solving in TcShortCut mode means /ignoring the Givens/. So we should not pass them to typechecker plugins! Fixes #26258. This is a fixup to the earlier MR: commit 1bd12371feacc52394a0e660ef9349f9e8ee1c06 Author: Simon Peyton Jones Date: Mon Jul 21 10:04:49 2025 +0100 Improve treatment of SPECIALISE pragmas -- again! - - - - - 2157db2d by sterni at 2025-08-08T15:32:39-04:00 hadrian: enable terminfo if --with-curses-* flags are given The GHC make build system used to support WITH_TERMINFO in ghc.mk which allowed controlling whether to build GHC with terminfo or not. hadrian has replaced this with a system where this is effectively controlled by the cross-compiling setting (the default WITH_TERMINFO value was bassed on CrossCompiling, iirc). This behavior is undesireable in some cases and there is not really a good way to work around it. Especially for downstream packagers, modifying this via UserSettings is not really feasible since such a source file has to be kept in sync with Settings/Default.hs manually since it can't import Settings.Default or any predefined Flavour definitions. To avoid having to add a new setting to cfg/system.config and/or a new configure flag (though I'm happy to implement both if required), I've chosen to take --with-curses-* being set explicitly as an indication that the user wants to have terminfo enabled. This would work for Nixpkgs which sets these flags [1] as well as haskell.nix [2] (which goes to some extreme measures [3] [4] to force terminfo in all scenarios). In general, I'm an advocate for making the GHC build be the same for native and cross insofar it is possible since it makes packaging GHC and Haskell related things while still supporting cross much less compilicated. A more minimal GHC with reduced dependencies should probably be a specific flavor, not the default. Partially addresses #26288 by forcing terminfo to be built if the user explicitly passes configure flags related to it. However, it isn't built by default when cross-compiling yet nor is there an explicit way to control the package being built. [1]: https://github.com/NixOS/nixpkgs/blob/3a7266fcefcb9ce353df49ba3f292d06443760... [2]: https://github.com/input-output-hk/haskell.nix/blob/6eaafcdf04bab7be745d1aa4... [3]: https://github.com/input-output-hk/haskell.nix/blob/6eaafcdf04bab7be745d1aa4... [4]: https://github.com/input-output-hk/haskell.nix/blob/6eaafcdf04bab7be745d1aa4... - - - - - b3c31488 by David Feuer at 2025-08-08T15:33:21-04:00 Add default QuasiQuoters Add `defaultQuasiQuoter` and `namedDefaultQuasiQuoter` to make it easier to write `QuasiQuoters` that give helpful error messages when they're used in inappropriate contexts. Closes #24434. - - - - - 9bf81834 by Rodrigo Mesquita at 2025-08-10T11:18:26+00:00 cleanup: Move dehydrateCgBreakInfo to Stg2Bc This no longer has anything to do with Core. - - - - - d6335551 by Rodrigo Mesquita at 2025-08-10T11:18:26+00:00 rts/Disassembler: Fix spacing of BRK_FUN - - - - - 4dd9b18b by Rodrigo Mesquita at 2025-08-10T11:18:27+00:00 debugger: Fix bciPtr in Step-out We need to use `BCO_NEXT` to move bciPtr to ix=1, because ix=0 points to the instruction itself! I do not understand how this didn't crash before. - - - - - a75fafd4 by Rodrigo Mesquita at 2025-08-10T11:18:27+00:00 debugger: Allow BRK_FUNs to head case continuation BCOs When we start executing a BCO, we may want to yield to the scheduler: this may be triggered by a heap/stack check, context switch, or a breakpoint. To yield, we need to put the stack in a state such that when execution is resumed we are back to where we yielded from. Previously, a BKR_FUN could only head a function BCO because we only knew how to construct a valid stack for yielding from one -- simply add `apply_interp_info` + the BCO to resume executing. This is valid because the stack at the start of run_BCO is headed by that BCO's arguments. However, in case continuation BCOs (as per Note [Case continuation BCOs]), we couldn't easily reconstruct a valid stack that could be resumed because we dropped too soon the stack frames regarding the value returned (stg_ret) and received (stg_ctoi) by that continuation. This is especially tricky because of the variable type and size return frames (e.g. pointer ret_p/ctoi_R1p vs a tuple ret_t/ctoi_t2). The trick to being able to yield from a BRK_FUN at the start of a case cont BCO is to stop removing the ret frame headers eagerly and instead keep them until the BCO starts executing. The new layout at the start of a case cont. BCO is described by the new Note [Stack layout when entering run_BCO]. Now, we keep the ret_* and ctoi_* frames when entering run_BCO. A BRK_FUN is then executed if found, and the stack is yielded as-is with the preserved ret and ctoi frames. Then, a case cont BCO's instructions always SLIDE off the headers of the ret and ctoi frames, in StgToByteCode.doCase, turning a stack like | .... | +---------------+ | fv2 | +---------------+ | fv1 | +---------------+ | BCO | +---------------+ | stg_ctoi_ret_ | +---------------+ | retval | +---------------+ | stg_ret_..... | +---------------+ into | .... | +---------------+ | fv2 | +---------------+ | fv1 | +---------------+ | retval | +---------------+ for the remainder of the BCO. Moreover, this more uniform approach of keeping the ret and ctoi frames means we need less ad-hoc logic concerning the variable size of ret_tuple vs ret_p/np frames in the code generator and interpreter: Always keep the return to cont. stack intact at the start of run_BCO, and the statically generated instructions will take care of adjusting it. Unlocks BRK_FUNs at the start of case cont. BCOs which will enable a better user-facing step-out (#26042) which is free of the bugs the current BRK_ALTS implementation suffers from (namely, using BRK_FUN rather than BRK_ALTS in a case cont. means we'll never accidentally end up in a breakpoint "deeper" than the continuation, because we stop at the case cont itself rather than on the first breakpoint we evaluate after it). - - - - - f03ef9d3 by Rodrigo Mesquita at 2025-08-10T11:18:27+00:00 BRK_FUN with InternalBreakLocs for code-generation time breakpoints At the start of a case continuation BCO, place a BRK_FUN. This BRK_FUN uses the new "internal breakpoint location" -- allowing us to come up with a valid source location for this breakpoint that is not associated with a source-level tick. For case continuation BCOs, we use the last tick seen before it as the source location. The reasoning is described in Note [Debugger: Stepout internal break locs]. Note how T26042c, which was broken because it displayed the incorrect behavior of the previous step out when we'd end up at a deeper level than the one from which we initiated step-out, is now fixed. As of this commit, BRK_ALTS is now dead code and is thus dropped. Note [Debugger: Stepout internal break locs] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Step-out tells the interpreter to run until the current function returns to where it was called from, and stop there. This is achieved by enabling the BRK_FUN found on the first RET_BCO frame on the stack (See [Note Debugger: Step-out]). Case continuation BCOs (which select an alternative branch) must therefore be headed by a BRK_FUN. An example: f x = case g x of <--- end up here 1 -> ... 2 -> ... g y = ... <--- step out from here - `g` will return a value to the case continuation BCO in `f` - The case continuation BCO will receive the value returned from g - Match on it and push the alternative continuation for that branch - And then enter that alternative. If we step-out of `g`, the first RET_BCO on the stack is the case continuation of `f` -- execution should stop at its start, before selecting an alternative. (One might ask, "why not enable the breakpoint in the alternative instead?", because the alternative continuation is only pushed to the stack *after* it is selected by the case cont. BCO) However, the case cont. BCO is not associated with any source-level tick, it is merely the glue code which selects alternatives which do have source level ticks. Therefore, we have to come up at code generation time with a breakpoint location ('InternalBreakLoc') to display to the user when it is stopped there. Our solution is to use the last tick seen just before reaching the case continuation. This is robust because a case continuation will thus always have a relevant breakpoint location: - The source location will be the last source-relevant expression executed before the continuation is pushed - So the source location will point to the thing you've just stepped out of - Doing :step-local from there will put you on the selected alternative (which at the source level may also be the e.g. next line in a do-block) Examples, using angle brackets (<<...>>) to denote the breakpoint span: f x = case <<g x>> {- step in here -} of 1 -> ... 2 -> ...> g y = <<...>> <--- step out from here ... f x = < ... 2 -> ...>> doing :step-local ... f x = case g x of 1 -> <<...>> <--- stop in the alternative 2 -> ... A second example based on T26042d2, where the source is a do-block IO action, optimised to a chain of `case expressions`. main = do putStrLn "hello1" <<f>> <--- step-in here putStrLn "hello3" putStrLn "hello4" f = do <> <--- step-out from here putStrLn "hello2.2" ... main = do putStrLn "hello1" <<f>> <--- end up here again, the previously executed expression putStrLn "hello3" putStrLn "hello4" doing step/step-local ... main = do putStrLn "hello1" f <> <--- straight to the next line putStrLn "hello4" Finishes #26042 - - - - - 52ba24df by Rodrigo Mesquita at 2025-08-10T11:18:27+00:00 debugger: Re-use the last BreakpointId whole in step-out Previously, to come up with a location to stop at for `:stepout`, we would store the location of the last BreakpointId surrounding the continuation, as described by Note [Debugger: Stepout internal break locs]. However, re-using just the location from the last source breakpoint isn't sufficient to provide the necessary information in the break location. Specifically, it wouldn't bind any variables at that location. Really, there is no reason not to re-use the last breakpoint wholesale, and re-use all the information we had there. Step-out should behave just as if we had stopped at the call, but s.t. continuing will not re-execute the call. This commit updates the CgBreakInfo to always store a BreakpointId, be it the original one or the one we're emulating (for step-out). It makes variable bindings on :stepout work - - - - - 0fe77f20 by Cheng Shao at 2025-08-10T11:18:27+00:00 compiler: WIP GHC.ByteCode.Serialize - - - - - 759209ae by Cheng Shao at 2025-08-10T11:18:27+00:00 driver: test bytecode roundtrip serialization - - - - - 72 changed files: - .gitlab/darwin/toolchain.nix - compiler/GHC/ByteCode/Asm.hs - compiler/GHC/ByteCode/Breakpoints.hs - compiler/GHC/ByteCode/Instr.hs - + compiler/GHC/ByteCode/Serialize.hs - compiler/GHC/Cmm.hs - compiler/GHC/CmmToAsm/PPC/Ppr.hs - compiler/GHC/CmmToAsm/Ppr.hs - compiler/GHC/CmmToLlvm/Data.hs - compiler/GHC/CoreToIface.hs - compiler/GHC/Driver/Main.hs - compiler/GHC/Linker/Loader.hs - compiler/GHC/Runtime/Debugger/Breakpoints.hs - compiler/GHC/Runtime/Eval.hs - compiler/GHC/StgToByteCode.hs - compiler/GHC/StgToCmm/InfoTableProv.hs - compiler/GHC/Tc/Gen/Export.hs - compiler/GHC/Tc/Solver/Dict.hs - compiler/GHC/Tc/Solver/Monad.hs - compiler/GHC/Tc/Solver/Solve.hs - compiler/GHC/Types/Name/Reader.hs - compiler/GHC/Utils/Binary.hs - compiler/ghc.cabal.in - docs/users_guide/debug-info.rst - ghc/GHCi/UI.hs - hadrian/src/Settings/Default.hs - hadrian/src/Settings/Packages.hs - libraries/ghc-internal/src/GHC/Internal/TH/Quote.hs - libraries/ghci/GHCi/Run.hs - libraries/template-haskell/Language/Haskell/TH/Quote.hs - libraries/template-haskell/changelog.md - rts/Disassembler.c - rts/IPE.c - rts/Interpreter.c - rts/ProfHeap.c - rts/Profiling.c - rts/eventlog/EventLog.c - rts/include/rts/Bytecodes.h - rts/include/rts/IPE.h - testsuite/tests/count-deps/CountDepsAst.stdout - testsuite/tests/count-deps/CountDepsParser.stdout - testsuite/tests/ghci.debugger/scripts/T26042b.script - testsuite/tests/ghci.debugger/scripts/T26042b.stdout - testsuite/tests/ghci.debugger/scripts/T26042c.script - testsuite/tests/ghci.debugger/scripts/T26042c.stdout - + testsuite/tests/ghci.debugger/scripts/T26042d2.hs - + testsuite/tests/ghci.debugger/scripts/T26042d2.script - + testsuite/tests/ghci.debugger/scripts/T26042d2.stdout - testsuite/tests/ghci.debugger/scripts/T26042e.stdout - testsuite/tests/ghci.debugger/scripts/T26042f.script - testsuite/tests/ghci.debugger/scripts/T26042f1.stdout - testsuite/tests/ghci.debugger/scripts/T26042f2.stdout - testsuite/tests/ghci.debugger/scripts/T26042g.stdout - testsuite/tests/ghci.debugger/scripts/all.T - testsuite/tests/interface-stability/template-haskell-exports.stdout - testsuite/tests/rts/ipe/ipeMap.c - testsuite/tests/rts/ipe/ipe_lib.c - + testsuite/tests/splice-imports/DodgyLevelExport.hs - + testsuite/tests/splice-imports/DodgyLevelExport.stderr - + testsuite/tests/splice-imports/DodgyLevelExportA.hs - + testsuite/tests/splice-imports/LevelImportExports.hs - + testsuite/tests/splice-imports/LevelImportExports.stdout - + testsuite/tests/splice-imports/LevelImportExportsA.hs - testsuite/tests/splice-imports/Makefile - + testsuite/tests/splice-imports/ModuleExport.hs - + testsuite/tests/splice-imports/ModuleExport.stderr - + testsuite/tests/splice-imports/ModuleExportA.hs - + testsuite/tests/splice-imports/ModuleExportB.hs - + testsuite/tests/splice-imports/T26090.hs - + testsuite/tests/splice-imports/T26090.stderr - + testsuite/tests/splice-imports/T26090A.hs - testsuite/tests/splice-imports/all.T The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/b0b466a55f49aecfdd59bf9d29fbe99... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/b0b466a55f49aecfdd59bf9d29fbe99... You're receiving this email because of your account on gitlab.haskell.org.