[Git][ghc/ghc][wip/az/ghc-cpp] 18 commits: compiler: Attempt to systematize Unique tags by introducing an ADT for each different tag

Alan Zimmerman pushed to branch wip/az/ghc-cpp at Glasgow Haskell Compiler / GHC Commits: f85058d3 by Zubin Duggal at 2025-10-17T13:50:52+05:30 compiler: Attempt to systematize Unique tags by introducing an ADT for each different tag Fixes #26264 Metric Decrease: T9233 - - - - - c85c845d by sheaf at 2025-10-17T22:35:32-04:00 Don't prematurely final-zonk PatSyn declarations This commit makes GHC hold off on the final zonk for pattern synonym declarations, in 'GHC.Tc.TyCl.PatSyn.tc_patsyn_finish'. This accommodates the fact that pattern synonym declarations without a type signature can contain unfilled metavariables, e.g. if the RHS of the pattern synonym involves view-patterns whose type mentions promoted (level 0) metavariables. Just like we do for ordinary function bindings, we should allow these metavariables to be settled later, instead of eagerly performing a final zonk-to-type. Now, the final zonking-to-type for pattern synonyms is performed in GHC.Tc.Module.zonkTcGblEnv. Fixes #26465 - - - - - ba3e5bdd by Rodrigo Mesquita at 2025-10-18T16:57:18-04:00 Move code-gen aux symbols from ghc-internal to rts These symbols were all previously defined in ghc-internal and made the dependency structure awkward, where the rts may refer to some of these symbols and had to work around that circular dependency the way described in #26166. Moreover, the code generator will produce code that uses these symbols! Therefore, they should be available in the rts: PRINCIPLE: If the code generator may produce code which uses this symbol, then it should be defined in the rts rather than, say, ghc-internal. That said, the main motivation is towards fixing #26166. Towards #26166. Pre-requisite of !14892 - - - - - f31de2a9 by Ben Gamari at 2025-10-18T16:57:18-04:00 rts: Avoid static symbol references to ghc-internal This resolves #26166, a bug due to new constraints placed by Apple's linker on undefined references. One source of such references in the RTS is the many symbols referenced in ghc-internal. To mitigate #26166, we make these references dynamic, as described in Note [RTS/ghc-internal interface]. Fixes #26166 Co-authored-by: Rodrigo Mesquita Co-authored-by: Cheng Shao - - - - - 43fdfddc by Ben Gamari at 2025-10-18T16:57:18-04:00 compiler: Rename isMathFun -> isLibcFun This set includes more than just math functions. - - - - - 4ed5138f by Ben Gamari at 2025-10-18T16:57:18-04:00 compiler: Add libc allocator functions to libc_funs Prototypes for these are now visible from `Prim.h`, resulting in multiple-declaration warnings in the unregisterised job. - - - - - 9a0a076b by Ben Gamari at 2025-10-18T16:57:18-04:00 rts: Minimize header dependencies of Prim.h Otherwise we will end up with redundant and incompatible declarations resulting in warnings during the unregisterised build. - - - - - 26b8a414 by Diego Antonio Rosario Palomino at 2025-10-18T16:58:10-04:00 Cmm Parser: Fix incorrect example in comment The Parser.y file contains a comment with an incorrect example of textual Cmm (used in .cmm files). This commit updates the comment to ensure it reflects valid textual Cmm syntax. Fixes #26313 - - - - - d4a9d6d6 by ARATA Mizuki at 2025-10-19T18:43:47+09:00 Handle implications between x86 feature flags This includes: * Multiple -msse* options can be specified * -mavx implies -msse4.2 * -mavx2 implies -mavx * -mfma implies -mavx * -mavx512f implies -mavx2 and -mfma * -mavx512{cd,er,pf} imply -mavx512f Closes #24989 Co-authored-by: sheaf - - - - - c9b8465c by Cheng Shao at 2025-10-20T10:16:00-04:00 wasm: workaround WebKit bug in dyld This patch works around a WebKit bug and allows dyld to run on WebKit based platforms as well. See added note for detailed explanation. Co-authored-by: Codex - - - - - 91b6be10 by Julian Ospald at 2025-10-20T18:21:03-04:00 Improve error handling in 'getPackageArchives' When the library dirs in the package conf files are not set up correctly, the JS linker will happily ignore such packages and not link against them, although they're part of the link plan. Fixes #26383 - - - - - 6c5269da by Sven Tennie at 2025-10-20T18:21:44-04:00 Align coding style Improve readability by using the same style for all constructor calls in this function. - - - - - 3d305889 by Sven Tennie at 2025-10-20T18:21:44-04:00 Reduce complexity by removing joins with mempty ldArgs, cArgs and cppArgs are all `mempty`. Thus concatenating them adds nothing but some complexity while reading the code. - - - - - 7e8605dd by Alan Zimmerman at 2025-10-22T19:34:22+01:00 GHC-CPP: Initial implementation Processes #define FOO #ifdef FOO x = 1 #endif Into [ITcppIgnored [L loc ITcppDefine] ,ITcppIgnored [L loc ITcppIfdef] ,ITvarid "x" ,ITequal ,ITinteger (IL {il_text = SourceText "1", il_neg = False, il_value = 1}) ,ITcppIgnored [L loc ITcppEndif] ,ITeof] In time, ITcppIgnored will be pushed into a comment Tidy up before re-visiting the continuation mechanic Switch preprocessor to continuation passing style Proof of concept, needs tidying up Small cleanup Get rid of some cruft Summary: Patch: Author: Alan Zimmerman Date: 2025-10-12 16:23:56 +0100 Summary: Patch: rebase-and-tests-pass Author: Alan Zimmerman Date: 2025-10-12 14:19:04 +0100 Rebase, and all tests pass except whitespace for generated parser Starting to integrate. Need to get the pragma recognised and set Make cppTokens extend to end of line, and process CPP comments Remove unused ITcppDefined Allow spaces between # and keyword for preprocessor directive Process CPP continuation lines They are emited as separate ITcppContinue tokens. Perhaps the processing should be more like a comment, and keep on going to the end. BUT, the last line needs to be slurped as a whole. Accumulate CPP continuations, process when ready Can be simplified further, we only need one CPP token Simplify Lexer interface. Only ITcpp We transfer directive lines through it, then parse them from scratch in the preprocessor. Deal with directive on last line, with no trailing \n Start parsing and processing the directives Prepare for processing include files Move PpState into PreProcess And initParserState, initPragState too Process nested include files Also move PpState out of Lexer.x, so it is easy to evolve it in a ghci session, loading utils/check-cpp/Main.hs Split into separate files Starting on expression parser. But it hangs. Time for Text.Parsec.Expr Start integrating the ghc-cpp work From https://github.com/alanz/ghc-cpp WIP Fixup after rebase WIP Fixup after rebase, including all tests pass Change pragma usage to GHC_CPP from GhcCPP Some comments Reformat Delete unused file Rename module Parse to ParsePP Clarify naming in the parser WIP. Switching to alex/happy to be able to work in-tree Since Parsec is not available Layering is now correct - GHC lexer, emits CPP tokens - accumulated in Preprocessor state - Lexed by CPP lexer, CPP command extracted, tokens concated with spaces (to get rid of token pasting via comments) - if directive lexed and parsed by CPP lexer/parser, and evaluated First example working Loading Example1.hs into ghci, getting the right results ``` {-# LANGUAGE GHC_CPP #-} module Example1 where y = 3 x = "hello" "bye now" foo = putStrLn x ``` Rebase, and all tests pass except whitespace for generated parser info: patch template saved to `-` More plumbing. Ready for testing tomorrow. Proress. Renamed module State from Types And at first blush it seems to handle preprocessor scopes properly. Insert basic GHC version macros into parser __GLASGOW_HASKELL__ __GLASGOW_HASKELL_FULL_VERSION__ __GLASGOW_HASKELL_PATCHLEVEL1__ __GLASGOW_HASKELL_PATCHLEVEL2__ Re-sync check-cpp for easy ghci work Get rid of warnings Rework macro processing, in check-cpp Macros kept at the top level, looked up via name, multiple arity versions per name can be stored WIP. Can crack arguments for #define Next step it to crack out args in an expansion WIP on arg parsing. Progress. Still screwing up nested parens. Seems to work, but has redundant code Remove redundant code Reformat Expand args, single pass Still need to repeat until fixpoint Fixed point expansion Sync the playground to compiler Working on dumping the GHC_CPP result But We need to keep the BufSpan in a comment Keep BufSpan in queued comments in GHC.Parser.Lexer Getting close to being able to print the combined tokens showing what is in and what is out First implementation of dumpGhcCpp. Example output First dumps all macros in the state, then the source, showing which lines are in and which are out ------------------------------ - |#define FOO(A,B) A + B - |#define FOO(A,B,C) A + B + C - |#if FOO(1,FOO(3,4)) == 8 - |-- a comment |x = 1 - |#else - |x = 5 - |#endif Clean up a bit Add -ddump-ghc-cpp option and a test based on it Restore Lexer.x rules, we need them for continuation lines Lexer.x: trying to sort out the span for continuations - We need to match on \n at the end of the line - We cannot simply back up for it Inserts predefined macros. But does not dump properly Because the cpp tokens have a trailing newline Remove unnecessary LExer rules We *need* the ones that explicitly match to the end of the line. Generate correct span for ITcpp Dump now works, except we do not render trailing `\` for continuation lines. This is good enough for use in test output. Reduce duplication in lexer Tweaks Insert min_version predefined macros into state The mechanism now works. Still need to flesh out the full set. Trying my alternative pragma syntax. It works, but dumpGhcCpp is broken, I suspect from the ITcpp token span update. Pragma extraction now works, with both CPP and GHC_CPP For the following {-# LANGUAGE CPP #-} #if __GLASGOW_HASKELL__ >= 913 {-# LANGUAGE GHC_CPP #-} #endif We will enable GHC_CPP only Remove some tracing Fix test exes for changes For GHC_CPP tests, normalise config-time-based macros WIP WIP again. What is wrong? Revert to dynflags for normal not pragma lexing Working on getting check-exact to work properly Passes CppCommentPlacement test Starting on exact printing with GHC_CPP While overriding normal CPP Correctly store CPP ignored tokens as comments By populating the lexeme string in it, based on the bufpos WIP Simplifying Update the active state logic Work the new logic into the mainline code Process `defined` operator Manage lexer state while skipping tokens There is very intricate layout-related state used when lexing. If a CPP directive blanks out some tokens, store this state when the blanking starts, and restore it when they are no longer being blanked. Track the last token buffer index, for ITCppIgnored We need to attach the source being skipped in an ITCppIgnored token. We cannot simply use its BufSpan as an index into the underlying StringBuffer as it counts unicode chars, not bytes. So we update the lexer state to store the starting StringBuffer location for the last token, and use the already-stored length to extract the correct portion of the StringBuffer being parsed. Process the ! operator in GHC_CPP expressions Predefine a constant when GHC_CPP is being used. WIP Skip lines directly in the lexer when required Properly manage location when accepting tokens again Seems to be working now, for Example9 Remove tracing Fix parsing '*' in block comments Instead of replacing them with '-' Keep the trailing backslash in a ITcpp token Deal with only enabling one section of a group. A group is an instance of a conditional introduced by #if/#ifdef/#ifndef, and ending at the final #endif, including intermediate #elsif sections Replace remaining identifiers with 0 when evaluating As per the spec Snapshot before rebase Skip non-processed lines starting with # Export generateMacros so we can use it in ghc-exactprint Fix rebase Expose initParserStateWithMacrosString Fix buggy lexer cppSkip It was skipping all lines, not just ones prefixed by # Fix evaluation of && to use the correct operator Deal with closing #-} at the start of a line Add the MIN_VERSION_GLASGOW_HASKELL predefined macro Include MIN_VERSION_GLASGOW_HASKELL in GhcCpp01.stderr Use a strict map for macro defines Process TIdentifierLParen Which only matters at the start of #define Do not provide TIdentifierLParen paren twice Handle whitespace between identifier and '(' for directive only Expose some Lexer bitmap manipulation helpers Deal with line pragmas as tokens Blows up for dumpGhcCpp though Allow strings delimited by a single quote too Allow leading whitespace on cpp directives As per https://timsong-cpp.github.io/cppwp/n4140/cpp#1 Implement GHC_CPP undef Sort out expansion of no-arg macros, in a context with args And make the expansion bottom out, in the case of recursion Fix GhcCpp01 test The LINE pragma stuff works in ghc-exactprint when specifically setting flag to emit ITline_pragma tokens Process comments in CPP directives Correctly lex pragmas with finel #-} on a newline Do not process CPP-style comments Allow cpp-style comments when GHC_CPP enabled Return other pragmas as cpp ignored when GHC_CPP active Reorganise getOptionsFromFile for use in ghc-exactprint We want to be able to inject predefined macro definitions into the parser preprocessor state for when we do a hackage roundtrip. Tweak testing Only allow unknown cpp pragmas with # in left margin Require # against left margin for all GHC_CPP directives Fix CPP directives appearing in pragmas And add a test for error reporting for missing `#if` Starting to report GHC_CPP errors using GHC machinery More GHC_CPP diagnostic results WIP on converting error calls to GHC diagnostics in GHC_CPP Working on CPP diagnostic reporting Tweak some tests/lint warnings More error reporting in Macro Some cleanups Some cleanup GHC_CPP: Working on improving error reporting Harvest some commonality Use PPM as Maybe inside PP Clean up a bit Fix GhcCpp01 test I think this needs to be made more robust. Likely by not dumping the (pre-)defined macros. info: patch template saved to `-` info: patch template saved to `-` - - - - - df524b7c by Alan Zimmerman at 2025-10-22T19:34:22+01:00 Fix GhcCpp01 after rebase - - - - - 8d2884c7 by Alan Zimmerman at 2025-10-22T19:34:22+01:00 GHC_CPP: delete utils/check-cpp It is not needed - - - - - a83161f1 by Alan Zimmerman at 2025-10-22T19:34:22+01:00 GHC_CPP: remove #include processing We do not support it - - - - - 2a73e7a4 by Alan Zimmerman at 2025-10-22T19:34:22+01:00 GHC_CPP: move Eval.hs into GHC.Parser.PreProcess.Macro - - - - - 184 changed files: - compiler/GHC.hs - compiler/GHC/Builtin/Uniques.hs - compiler/GHC/Cmm/CLabel.hs - compiler/GHC/Cmm/Info.hs - compiler/GHC/Cmm/Info/Build.hs - compiler/GHC/Cmm/Lexer.x - compiler/GHC/Cmm/Parser.y - compiler/GHC/Cmm/Parser/Monad.hs - compiler/GHC/Cmm/Pipeline.hs - compiler/GHC/Cmm/UniqueRenamer.hs - compiler/GHC/CmmToAsm/Config.hs - compiler/GHC/CmmToAsm/Wasm/FromCmm.hs - compiler/GHC/CmmToAsm/X86/CodeGen.hs - compiler/GHC/CmmToAsm/X86/Instr.hs - compiler/GHC/CmmToC.hs - compiler/GHC/CmmToLlvm/Base.hs - compiler/GHC/Core/Make.hs - compiler/GHC/Core/Opt/Monad.hs - compiler/GHC/Core/Opt/Pipeline.hs - compiler/GHC/Core/Opt/Simplify/Monad.hs - compiler/GHC/Core/PatSyn.hs - compiler/GHC/CoreToStg/Prep.hs - compiler/GHC/Driver/Backpack.hs - compiler/GHC/Driver/CodeOutput.hs - compiler/GHC/Driver/Config/CmmToAsm.hs - compiler/GHC/Driver/Config/Parser.hs - compiler/GHC/Driver/Downsweep.hs - compiler/GHC/Driver/DynFlags.hs - compiler/GHC/Driver/Flags.hs - compiler/GHC/Driver/Main.hs - compiler/GHC/Driver/Pipeline.hs - compiler/GHC/Driver/Pipeline/Execute.hs - compiler/GHC/Driver/Session.hs - compiler/GHC/HsToCore/Foreign/C.hs - compiler/GHC/HsToCore/Foreign/JavaScript.hs - compiler/GHC/HsToCore/Foreign/Wasm.hs - compiler/GHC/HsToCore/Monad.hs - compiler/GHC/HsToCore/Utils.hs - compiler/GHC/Iface/Binary.hs - compiler/GHC/Iface/Rename.hs - compiler/GHC/JS/JStg/Monad.hs - compiler/GHC/Linker/Dynamic.hs - compiler/GHC/Linker/Static.hs - compiler/GHC/Parser.hs-boot - compiler/GHC/Parser.y - compiler/GHC/Parser/Annotation.hs - compiler/GHC/Parser/Errors/Ppr.hs - compiler/GHC/Parser/Errors/Types.hs - compiler/GHC/Parser/HaddockLex.x - compiler/GHC/Parser/Header.hs - compiler/GHC/Parser/Lexer.x - compiler/GHC/Parser/PostProcess.hs - compiler/GHC/Parser/PostProcess/Haddock.hs - + compiler/GHC/Parser/PreProcess.hs - + compiler/GHC/Parser/PreProcess/Lexer.x - + compiler/GHC/Parser/PreProcess/Macro.hs - + compiler/GHC/Parser/PreProcess/ParsePP.hs - + compiler/GHC/Parser/PreProcess/Parser.y - + compiler/GHC/Parser/PreProcess/ParserM.hs - + compiler/GHC/Parser/PreProcess/State.hs - compiler/GHC/Parser/Utils.hs - compiler/GHC/Platform.hs - compiler/GHC/Platform/Reg.hs - compiler/GHC/Runtime/Eval.hs - compiler/GHC/Stg/EnforceEpt.hs - compiler/GHC/Stg/Pipeline.hs - compiler/GHC/StgToCmm/ExtCode.hs - compiler/GHC/StgToCmm/Monad.hs - compiler/GHC/StgToJS/CodeGen.hs - compiler/GHC/StgToJS/Ids.hs - compiler/GHC/StgToJS/Linker/Linker.hs - compiler/GHC/SysTools/Cpp.hs - compiler/GHC/Tc/Module.hs - compiler/GHC/Tc/TyCl/PatSyn.hs - compiler/GHC/Tc/Types.hs - compiler/GHC/Tc/Types/Origin.hs - compiler/GHC/Tc/Utils/Monad.hs - compiler/GHC/Tc/Zonk/Type.hs - compiler/GHC/Types/Error/Codes.hs - compiler/GHC/Types/Name/Cache.hs - compiler/GHC/Types/Unique.hs - compiler/GHC/Types/Unique/DSM.hs - compiler/GHC/Types/Unique/Supply.hs - compiler/Language/Haskell/Syntax/Binds.hs - compiler/ghc.cabal.in - docs/users_guide/9.16.1-notes.rst - docs/users_guide/debugging.rst - docs/users_guide/using.rst - ghc/GHCi/UI.hs - hadrian/src/Rules/Gmp.hs - hadrian/src/Rules/Libffi.hs - hadrian/src/Rules/SourceDist.hs - hadrian/src/Settings/Builders/Cabal.hs - hadrian/src/Settings/Builders/Common.hs - hadrian/src/Settings/Builders/DeriveConstants.hs - hadrian/src/Settings/Builders/Hsc2Hs.hs - hadrian/src/Settings/Packages.hs - + libraries/ghc-internal/cbits/RtsIface.c - libraries/ghc-internal/ghc-internal.cabal.in - + libraries/ghc-internal/include/RtsIfaceSymbols.h - libraries/ghc-internal/src/GHC/Internal/LanguageExtensions.hs - rts/BuiltinClosures.c - rts/CloneStack.h - rts/Compact.cmm - rts/ContinuationOps.cmm - rts/Exception.cmm - rts/Prelude.h - rts/PrimOps.cmm - rts/RtsAPI.c - rts/RtsStartup.c - rts/RtsSymbols.c - + rts/RtsToHsIface.c - rts/StgStdThunks.cmm - rts/configure.ac - − rts/external-symbols.list.in - rts/include/Rts.h - rts/include/RtsAPI.h - rts/include/Stg.h - + rts/include/rts/RtsToHsIface.h - rts/include/rts/Types.h - rts/include/stg/Prim.h - rts/posix/Signals.c - libraries/ghc-internal/cbits/atomic.c → rts/prim/atomic.c - libraries/ghc-internal/cbits/bitrev.c → rts/prim/bitrev.c - libraries/ghc-internal/cbits/bswap.c → rts/prim/bswap.c - libraries/ghc-internal/cbits/clz.c → rts/prim/clz.c - libraries/ghc-internal/cbits/ctz.c → rts/prim/ctz.c - libraries/ghc-internal/cbits/int64x2minmax.c → rts/prim/int64x2minmax.c - libraries/ghc-internal/cbits/longlong.c → rts/prim/longlong.c - libraries/ghc-internal/cbits/mulIntMayOflo.c → rts/prim/mulIntMayOflo.c - libraries/ghc-internal/cbits/pdep.c → rts/prim/pdep.c - libraries/ghc-internal/cbits/pext.c → rts/prim/pext.c - libraries/ghc-internal/cbits/popcnt.c → rts/prim/popcnt.c - libraries/ghc-internal/cbits/vectorQuotRem.c → rts/prim/vectorQuotRem.c - libraries/ghc-internal/cbits/word2float.c → rts/prim/word2float.c - − rts/rts.buildinfo.in - rts/rts.cabal - rts/wasm/JSFFI.c - rts/wasm/scheduler.cmm - rts/win32/libHSghc-internal.def - testsuite/tests/codeGen/should_gen_asm/all.T - + testsuite/tests/codeGen/should_gen_asm/mavx-should-enable-popcnt.asm - + testsuite/tests/codeGen/should_gen_asm/mavx-should-enable-popcnt.hs - + testsuite/tests/codeGen/should_gen_asm/msse-option-order.asm - + testsuite/tests/codeGen/should_gen_asm/msse-option-order.hs - testsuite/tests/count-deps/CountDepsParser.stdout - testsuite/tests/driver/T4437.hs - testsuite/tests/ghc-api/T11579.hs - + testsuite/tests/ghc-api/T26264.hs - + testsuite/tests/ghc-api/T26264.stdout - testsuite/tests/ghc-api/all.T - + testsuite/tests/ghc-cpp/GhcCpp01.hs - + testsuite/tests/ghc-cpp/GhcCpp01.stderr - + testsuite/tests/ghc-cpp/GhcCpp02.hs - + testsuite/tests/ghc-cpp/GhcCpp02.stderr - + testsuite/tests/ghc-cpp/all.T - testsuite/tests/interface-stability/template-haskell-exports.stdout - testsuite/tests/linters/notes.stdout - testsuite/tests/linters/regex-linters/check-cpp.py - testsuite/tests/overloadedrecflds/should_compile/BootFldReexport.stderr - testsuite/tests/overloadedrecflds/should_fail/T16745.stderr - testsuite/tests/overloadedrecflds/should_fail/T18999_NoDisambiguateRecordFields.stderr - + testsuite/tests/patsyn/should_compile/T26465b.hs - + testsuite/tests/patsyn/should_compile/T26465c.hs - + testsuite/tests/patsyn/should_compile/T26465d.hs - + testsuite/tests/patsyn/should_compile/T26465d.stderr - testsuite/tests/patsyn/should_compile/all.T - + testsuite/tests/patsyn/should_fail/T26465.hs - + testsuite/tests/patsyn/should_fail/T26465.stderr - testsuite/tests/patsyn/should_fail/all.T - testsuite/tests/perf/should_run/UniqLoop.hs - + testsuite/tests/printer/CppCommentPlacement.hs - testsuite/tests/regalloc/regalloc_unit_tests.hs - testsuite/tests/simplCore/should_compile/rule2.stderr - testsuite/tests/th/T8761.stderr - utils/check-exact/Main.hs - utils/check-exact/Parsers.hs - utils/check-exact/Preprocess.hs - utils/check-exact/Utils.hs - utils/deriveConstants/Main.hs - utils/haddock/haddock-api/src/Haddock/Backends/Hyperlinker/Parser.hs - utils/haddock/haddock-api/src/Haddock/Parser.hs - utils/haddock/haddock-api/src/Haddock/Types.hs - utils/jsffi/dyld.mjs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a947e98700317c67a46614f74625bbe... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a947e98700317c67a46614f74625bbe... You're receiving this email because of your account on gitlab.haskell.org.