Sven Tennie pushed to branch wip/romes/hadrian-cross-stage2-rebase_SVEN_FINAL at Glasgow Haskell Compiler / GHC

Commits:

• e4a70020
  by Matthew Pickering at 2025-12-22T13:27:46+01:00

  hadrian: Build stage 2 cross compilers
  
  * Most of hadrian is abstracted over the stage in order to remove the
    assumption that the target of all stages is the same platform. This
    allows the RTS to be built for two different targets for example.
  * Abstracts the bindist creation logic to allow building either normal
    or cross bindists. Normal bindists use stage 1 libraries and a stage 2
    compiler. Cross bindists use stage 2 libararies and a stage 2
    compiler.
  * hadrian: Make binary-dist-dir the default build target. This allows us
    to have the logic in one place about which libraries/stages to build
    with cross compilers. Fixes #24192
  
  New hadrian target:
  
  * `binary-dist-dir-cross`: Build a cross compiler bindist (compiler =
    stage 1, libraries = stage 2)
  
  -------------------------
  Metric Decrease:
      T10421a
      T10858
      T11195
      T11276
      T11374
      T11822
      T15630
      T17096
      T18478
      T20261
  Metric Increase:
      parsing001
  -------------------------
  
  Fix hardcoded stage1
  
  Don't recache
  
  Additional SIMD flags are required for the host
  
  The files with specific SIMD flags are built for GHC's RTS (host), not
  for the programs built by it (target.) This matters when
  cross-compiling, because host and target differ then.
  
  Split up system.config into host/target config files
  
  There were a number of settings which were not applied per-stage, for
  example if you specified `--ffi-include-dir` then that was applied to
  both host and target. Now this will just be passed when building the
  crosscompiler.
  
  The solution for now is to separate these two files into host/target and
  the host file contains very bare-bones . There isn't currently a way to
  specify with configure anything in the host file, so if you are building
  a cross-compiler and you need to do that, you have to modify the file
  yourself.
  
  Fix location of emsdk-version
  
  fix distrib/configure file
  
  hadrian: Make text_simdutf flavour transformer configurable per-stage
  
  Before it was globally enabled, which was probably not what you want as
  you don't need text-simd for your boot compiler nor your boot compiler
  if you're building a cross-compiler.
  
  This brings it into line with the other modifiers.. such as ghcProfiled
  etc
  
  Fixes #25302
  
  fixes for simdutf8
  
  Hard-code ways in settings
  
  Fix ghcconfig lookup error
  
  This seems to be the fix with least friction for the issue stated below.
  Though, in the long run it might be better to rename `TargetARCH_CPP` to
  `TargetARCH` (the `_CPP` suffix feels a bit odd.)
  
  Fixed error:
  ```
  Key 'TargetARCH' not found in file '_build/test/ghcconfig'
  ```
  
  target-has-libm -> use-lib-m
  
  The flag was renamed.
  
  Fix path stage segment to stage mapping in generated rules
  
  Cleanup unused imports
  
  Fix out-of-tree TestCompilerArgs parsing: WORDSIZE
  
  TestWORDSIZE is in bits, not bytes.
  
  TestCompilerArgs: Fix arch (out of tree)
  
  Calculate "RTS ways"
  
  The static string doesn't reflect what GHC provides in tests.
  
  Fix libffi configuration
  
  Libffi needs to be built with the config of the successor stage.
  
  Fix libffi ghcjs
  
  hadrian: Fix predicate for building shared libraries in defaultLibraries
  
  Obviously we should only attempt to build shared libraries if the target
  supports building shared libraries.
  
  use building for target in llvm flavour transformer
  
  WIP: libffi: LD, OBJDUMP, STRIP staged
  
  Removing the env variables implies using the programs from $PATH. This
  kind-of works, but these values should be correctly auto-configured.
  
  Make stage2 cross windows build work - somehow
  
  Still needs some improvements.
  
  Adjust host_fully_static for stage2 cross builds
  
  Reference correct package.conf.d for cross
  
  Fixup Rebase
  
  Fixup: Align Settings
  
  Rebase fixup
  
  Libffi - no LD for cross host stages
  
  NOSMP has to be a C flag for RTS
  
  Otherwise building unregisterised fails. This is also in line with
  master.
  
  Rebase fixup: ghcWithInternalInterpreter
  
  Prevent settings file creation exception
  
  Do not try to reach out to settings beyond finalStage. That cannot work.
  
  Fix Asan
  

• 09edd16f
  by Sven Tennie at 2025-12-22T13:27:47+01:00

  Align CI scripts with master
  
  Fixup
  

• 1a37320a
  by Matthew Pickering at 2025-12-22T13:27:47+01:00

  ci: Test cross bindists
  
  We remove the special logic for testing in-tree cross
  compilers and instead test cross compiler bindists, like we do for all
  other platforms.
  

• 6aee82c9
  by Matthew Pickering at 2025-12-22T13:27:47+01:00

  ci: Javascript don't set CROSS_EMULATOR
  
  There is no CROSS_EMULATOR needed to run javascript binaries, so we
  don't set the CROSS_EMULATOR to some dummy value.
  

• 1d98d10b
  by Matthew Pickering at 2025-12-22T13:27:47+01:00

  ci: Introduce CROSS_STAGE variable
  
  In preparation for building and testing stage3 bindists we introduce the
  CROSS_STAGE variable which is used by a CI job to determine what kind of
  bindist the CI job should produce.
  
  At the moment we are only using CROSS_STAGE=2 but in the future we will
  have some jobs which set CROSS_STAGE=3 to produce native bindists for a
  target, but produced by a cross compiler, which can be tested on by
  another CI job on the native platform.
  
  CROSS_STAGE=2: Build a normal cross compiler bindist
  CROSS_STAGE=3: Build a stage 3 bindist, one which is a native compiler and library for the target
  

• 51c4f4c8
  by Matthew Pickering at 2025-12-22T13:27:47+01:00

  hadrian: Refactor system-cxx-std-lib rules0
  
  I noticed a few things wrong with the hadrian rules for `system-cxx-std-lib` rules.
  
  * For `text` there is an ad-hoc check to depend on `system-cxx-std-lib` outside of `configurePackage`.
  * The `system-cxx-std-lib` dependency is not read from cabal files.
  * Recache is not called on the packge database after the `.conf` file is generated, a more natural place for this rule is `registerRules`.
  
  Treating this uniformly like other packages is complicated by it not having any source code or a cabal file. However we can do a bit better by reporting the dependency firstly in `PackageData` and then needing the `.conf` file in the same place as every other package in `configurePackage`.
  
  Fixes #25303
  

• e0568f84
  by Matthew Pickering at 2025-12-22T13:27:47+01:00

  bindist: Pass path to package database we want to recache
  
  This fixes recaching on cross compilers
  

• a5b30720
  by Sven Tennie at 2025-12-22T13:27:47+01:00

  Increase timeout for emulators
  
  Test runs with emulators naturally take longer than on native machines.
  

• 55f080d8
  by Sven Tennie at 2025-12-22T13:27:47+01:00

  Provide config.cross flag to testsuite
  

• caa3ed67
  by Sven Tennie at 2025-12-22T13:27:47+01:00

  Distinguish between having an interpreter and having an internal one
  
  Otherwise, we fail with warnings when compiling tools.
  

68 changed files:

• .gitlab/ci.sh
• .gitlab/generate-ci/gen_ci.hs
• .gitlab/jobs.yaml
• configure.ac
• distrib/configure.ac.in
• ghc/GHC/Driver/Session/Mode.hs
• ghc/GHCi/UI.hs
• ghc/Main.hs
• ghc/ghc-bin.cabal.in
• hadrian/README.md
• hadrian/bindist/config.mk.in
• + hadrian/cfg/system.config.host.in
• hadrian/cfg/system.config.in
• + hadrian/cfg/system.config.target.in
• hadrian/hadrian.cabal
• hadrian/src/Base.hs
• + hadrian/src/BindistConfig.hs
• hadrian/src/Builder.hs
• hadrian/src/Context.hs
• hadrian/src/Expression.hs
• hadrian/src/Flavour.hs
• hadrian/src/Flavour/Type.hs
• hadrian/src/Hadrian/Builder.hs
• hadrian/src/Hadrian/Expression.hs
• hadrian/src/Hadrian/Haskell/Cabal/Parse.hs
• hadrian/src/Hadrian/Haskell/Cabal/Type.hs
• hadrian/src/Hadrian/Haskell/Hash.hs
• hadrian/src/Hadrian/Oracles/TextFile.hs
• hadrian/src/Oracles/Flag.hs
• hadrian/src/Oracles/Flavour.hs
• hadrian/src/Oracles/Setting.hs
• hadrian/src/Oracles/TestSettings.hs
• hadrian/src/Packages.hs
• hadrian/src/Rules.hs
• hadrian/src/Rules/BinaryDist.hs
• hadrian/src/Rules/CabalReinstall.hs
• hadrian/src/Rules/Compile.hs
• hadrian/src/Rules/Documentation.hs
• hadrian/src/Rules/Generate.hs
• hadrian/src/Rules/Gmp.hs
• hadrian/src/Rules/Libffi.hs
• hadrian/src/Rules/Library.hs
• hadrian/src/Rules/Program.hs
• hadrian/src/Rules/Register.hs
• hadrian/src/Rules/Rts.hs
• hadrian/src/Rules/Test.hs
• hadrian/src/Settings.hs
• hadrian/src/Settings/Builders/Cabal.hs
• hadrian/src/Settings/Builders/Common.hs
• hadrian/src/Settings/Builders/Configure.hs
• hadrian/src/Settings/Builders/DeriveConstants.hs
• hadrian/src/Settings/Builders/Ghc.hs
• hadrian/src/Settings/Builders/Hsc2Hs.hs
• hadrian/src/Settings/Builders/RunTest.hs
• hadrian/src/Settings/Builders/SplitSections.hs
• hadrian/src/Settings/Default.hs
• hadrian/src/Settings/Flavours/Benchmark.hs
• hadrian/src/Settings/Flavours/Development.hs
• hadrian/src/Settings/Flavours/GhcInGhci.hs
• hadrian/src/Settings/Flavours/Performance.hs
• hadrian/src/Settings/Flavours/Quick.hs
• hadrian/src/Settings/Flavours/QuickCross.hs
• hadrian/src/Settings/Flavours/Quickest.hs
• hadrian/src/Settings/Flavours/Validate.hs
• hadrian/src/Settings/Packages.hs
• hadrian/src/Settings/Program.hs
• hadrian/src/Settings/Warnings.hs
• testsuite/ghc-config/ghc-config.hs

The diff was not included because it is too large.