Sven Tennie pushed to branch wip/romes/hadrian-cross-stage2-rebase_SVEN_FIXED at Glasgow Haskell Compiler / GHC Commits: 8f3d80ff by Luite Stegeman at 2025-09-13T08:43:09+02:00 Use mkVirtHeapOffsets for reconstructing terms in RTTI This makes mkVirtHeapOffsets the single source of truth for finding field offsets in closures. - - - - - eb389338 by Luite Stegeman at 2025-09-13T08:43:09+02:00 Sort non-pointer fields by size for more efficient packing This sorts non-pointer fields in mkVirtHeapOffsets, always storing the largest field first. The relative order of equally sized fields remains unchanged. This reduces wasted padding/alignment space in closures with differently sized fields. - - - - - 99b233f4 by Alison at 2025-09-13T16:51:04-04:00 ghc-heap: Fix race condition with profiling builds Apply the same fix from Closures.hs (64fd0fac83) to Heap.hs by adding empty imports to make way-dependent dependencies visible to `ghc -M`. Fixes #15197, #26407 - - - - - 77deaa7a by Cheng Shao at 2025-09-14T21:29:45-04:00 hadrian: build in-tree gmp with -fvisibility=hidden When hadrian builds in-tree gmp, it should build the shared objects with -fvisibility=hidden. The gmp symbols are only used by bignum logic in ghc-internal and shouldn't be exported by the ghc-internal shared library. We should always strive to keep shared library symbol table lean, which benefits platforms with slow dynamic linker or even hard limits about how many symbols can be exported (e.g. macos dyld, win32 dll and wasm dyld). - - - - - 42a18960 by Cheng Shao at 2025-09-14T21:30:26-04:00 Revert "wasm: add brotli compression for ghci browser mode" This reverts commit 731217ce68a1093b5f9e26a07d5bd2cdade2b352. Benchmarks show non-negligible overhead when browser runs on the same host, which is the majority of actual use cases. - - - - - e6755b9f by Cheng Shao at 2025-09-14T21:30:26-04:00 wasm: remove etag logic in ghci browser mode web server This commit removes the etag logic in dyld script's ghci browser mode web server. It was meant to support caching logic of wasm shared libraries, but even if the port is manually specified to make caching even relevant, for localhost the extra overhead around etag logic is simply not worth it according to benchmarks. - - - - - 6ba841ef by Matthew Pickering at 2025-09-16T09:07:44+00:00 Add missing req_interp modifier to T18441fail3 and T18441fail19 These tests require the interpreter but they were failing in a different way with the javascript backend because the interpreter was disabled and stderr is ignored by the test. - - - - - 3f9decef by Matthew Pickering at 2025-09-16T09:07:44+00:00 Use explicit syntax rather than pure - - - - - 535f44b3 by Matthew Pickering at 2025-09-16T09:07:44+00:00 packaging: correctly propagate build/host/target to bindist configure script At the moment the host and target which we will produce a compiler for is fixed at the initial configure time. Therefore we need to persist the choice made at this time into the installation bindist as well so we look for the right tools, with the right prefixes at install time. In the future, we want to provide a bit more control about what kind of bindist we produce so the logic about what the host/target will have to be written by hadrian rather than persisted by the configure script. In particular with cross compilers we want to either build a normal stage 2 cross bindist or a stage 3 bindist, which creates a bindist which has a native compiler for the target platform. Fixes #21970 - - - - - dcb78b21 by Matthew Pickering at 2025-09-16T09:07:44+00:00 hadrian: Fill in more of the default.host toolchain file When you are building a cross compiler this file will be used to build stage1 and it's libraries, so we need enough information here to work accurately. There is still more work to be done (see for example, word size is still fixed). - - - - - c3ca6b53 by Matthew Pickering at 2025-09-16T09:07:44+00:00 hadrian: Disable docs when cross compiling Before there were a variety of ad-hoc places where doc building was disabled when cross compiling. * Some CI jobs sets --docs=none in gen_ci.hs * Some CI jobs set --docs=none in .gitlab/ci.sh * There was some logic in hadrian to not need the ["docs"] target when making a bindist. Now the situation is simple: * If you are cross compiling then defaultDocsTargets is empty by default. In theory, there is no reason why we can't build documentation for cross compiler bindists, but this is left to future work to generalise the documentation building rules to allow this (#24289) - - - - - facf4747 by Matthew Pickering at 2025-09-16T09:08:48+00:00 hadrian: Build stage 2 cross compilers * Most of hadrian is abstracted over the stage in order to remove the assumption that the target of all stages is the same platform. This allows the RTS to be built for two different targets for example. * Abstracts the bindist creation logic to allow building either normal or cross bindists. Normal bindists use stage 1 libraries and a stage 2 compiler. Cross bindists use stage 2 libararies and a stage 2 compiler. * hadrian: Make binary-dist-dir the default build target. This allows us to have the logic in one place about which libraries/stages to build with cross compilers. Fixes #24192 New hadrian target: * `binary-dist-dir-cross`: Build a cross compiler bindist (compiler = stage 1, libraries = stage 2) ------------------------- Metric Decrease: T10421a T10858 T11195 T11276 T11374 T11822 T15630 T17096 T18478 T20261 Metric Increase: parsing001 ------------------------- - - - - - 67f4d370 by Matthew Pickering at 2025-09-16T09:08:48+00:00 ci: Test cross bindists We remove the special logic for testing in-tree cross compilers and instead test cross compiler bindists, like we do for all other platforms. - - - - - e83297e8 by Matthew Pickering at 2025-09-16T09:08:48+00:00 ci: Javascript don't set CROSS_EMULATOR There is no CROSS_EMULATOR needed to run javascript binaries, so we don't set the CROSS_EMULATOR to some dummy value. - - - - - 27cc14d0 by Matthew Pickering at 2025-09-16T09:08:48+00:00 ci: Introduce CROSS_STAGE variable In preparation for building and testing stage3 bindists we introduce the CROSS_STAGE variable which is used by a CI job to determine what kind of bindist the CI job should produce. At the moment we are only using CROSS_STAGE=2 but in the future we will have some jobs which set CROSS_STAGE=3 to produce native bindists for a target, but produced by a cross compiler, which can be tested on by another CI job on the native platform. CROSS_STAGE=2: Build a normal cross compiler bindist CROSS_STAGE=3: Build a stage 3 bindist, one which is a native compiler and library for the target - - - - - d5722334 by Matthew Pickering at 2025-09-16T09:08:48+00:00 Split up system.config into host/target config files There were a number of settings which were not applied per-stage, for example if you specified `--ffi-include-dir` then that was applied to both host and target. Now this will just be passed when building the crosscompiler. The solution for now is to separate these two files into host/target and the host file contains very bare-bones . There isn't currently a way to specify with configure anything in the host file, so if you are building a cross-compiler and you need to do that, you have to modify the file yourself. - - - - - 9f11b1d7 by Matthew Pickering at 2025-09-16T09:08:48+00:00 wip fixup - - - - - cb22b2cb by Matthew Pickering at 2025-09-16T09:08:48+00:00 Fix location of emsdk-version - - - - - 509cb65f by Matthew Pickering at 2025-09-16T09:08:48+00:00 fix distrib/configure file - - - - - f3c776d8 by Matthew Pickering at 2025-09-16T09:08:48+00:00 Fix hardcoded stage1 - - - - - dc6a7fc5 by Matthew Pickering at 2025-09-16T09:08:48+00:00 Don't recache - - - - - 8850b414 by Matthew Pickering at 2025-09-16T09:08:48+00:00 hadrian: Make text_simdutf flavour transformer configurable per-stage Before it was globally enabled, which was probably not what you want as you don't need text-simd for your boot compiler nor your boot compiler if you're building a cross-compiler. This brings it into line with the other modifiers.. such as ghcProfiled etc Fixes #25302 - - - - - fed579a0 by Matthew Pickering at 2025-09-16T09:08:49+00:00 hadrian: Refactor system-cxx-std-lib rules0 I noticed a few things wrong with the hadrian rules for `system-cxx-std-lib` rules. * For `text` there is an ad-hoc check to depend on `system-cxx-std-lib` outside of `configurePackage`. * The `system-cxx-std-lib` dependency is not read from cabal files. * Recache is not called on the packge database after the `.conf` file is generated, a more natural place for this rule is `registerRules`. Treating this uniformly like other packages is complicated by it not having any source code or a cabal file. However we can do a bit better by reporting the dependency firstly in `PackageData` and then needing the `.conf` file in the same place as every other package in `configurePackage`. Fixes #25303 - - - - - fad01bdb by Matthew Pickering at 2025-09-16T09:08:49+00:00 fixes for simdutf8 - - - - - 802c36e3 by Matthew Pickering at 2025-09-16T09:08:49+00:00 use building for target in llvm flavour transformer - - - - - 95dc780c by Matthew Pickering at 2025-09-16T09:08:49+00:00 bindist: Pass path to package database we want to recache This fixes recaching on cross compilers - - - - - ee2a38b2 by Matthew Pickering at 2025-09-16T09:08:49+00:00 testsuite: T9930fail now passes on javascript I didn't investigate why, but the comment says it should be fixed by building a stage2 cross compiler (and it is). - - - - - 00e38c75 by Matthew Pickering at 2025-09-16T09:08:49+00:00 hadrian: Fix predicate for building shared libraries in defaultLibraries Obviously we should only attempt to build shared libraries if the target supports building shared libraries. - - - - - 918ca218 by Matthew Pickering at 2025-09-16T09:08:49+00:00 Hard-code ways in settings - - - - - 89e6741c by Sven Tennie at 2025-09-16T09:08:49+00:00 Fix ghcconfig lookup error This seems to be the fix with least friction for the issue stated below. Though, in the long run it might be better to rename `TargetARCH_CPP` to `TargetARCH` (the `_CPP` suffix feels a bit odd.) Fixed error: ``` Key 'TargetARCH' not found in file '_build/test/ghcconfig' ``` - - - - - ba5dfb23 by Sven Tennie at 2025-09-16T09:08:49+00:00 target-has-libm -> use-lib-m The flag was renamed. - - - - - 3ea66f9b by Sven Tennie at 2025-09-16T09:08:49+00:00 Additional SIMD flags are required for the host The files with specific SIMD flags are built for GHC's RTS (host), not for the programs built by it (target.) This matters when cross-compiling, because host and target differ then. - - - - - 4b30b536 by Sven Tennie at 2025-09-16T09:08:49+00:00 Fix path stage segment to stage mapping in generated rules - - - - - 00bf98ff by Sven Tennie at 2025-09-16T09:08:49+00:00 Cleanup: Delete unused binding - - - - - 6e0e3087 by Sven Tennie at 2025-09-16T09:08:49+00:00 Cleanup unused imports - - - - - 9ab61971 by Sven Tennie at 2025-09-16T09:08:49+00:00 Fix out-of-tree TestCompilerArgs parsing: WORDSIZE TestWORDSIZE is in bits, not bytes. - - - - - 7099c9bf by Sven Tennie at 2025-09-16T09:08:49+00:00 TestCompilerArgs: Fix arch (out of tree) - - - - - dfd48223 by Sven Tennie at 2025-09-16T09:08:49+00:00 Calculate "RTS ways" The static string doesn't reflect what GHC provides in tests. - - - - - 75 changed files: - .gitlab/ci.sh - .gitlab/generate-ci/gen_ci.hs - .gitlab/jobs.yaml - compiler/GHC/Runtime/Heap/Inspect.hs - compiler/GHC/StgToCmm/Layout.hs - configure.ac - distrib/configure.ac.in - hadrian/README.md - hadrian/bindist/config.mk.in - hadrian/cfg/default.host.target.in - + hadrian/cfg/system.config.host.in - hadrian/cfg/system.config.in - + hadrian/cfg/system.config.target.in - hadrian/hadrian.cabal - hadrian/src/Base.hs - + hadrian/src/BindistConfig.hs - hadrian/src/Builder.hs - hadrian/src/Context.hs - hadrian/src/Expression.hs - hadrian/src/Flavour.hs - hadrian/src/Flavour/Type.hs - hadrian/src/Hadrian/Expression.hs - hadrian/src/Hadrian/Haskell/Cabal/Parse.hs - hadrian/src/Hadrian/Haskell/Cabal/Type.hs - hadrian/src/Hadrian/Haskell/Hash.hs - hadrian/src/Hadrian/Oracles/TextFile.hs - hadrian/src/Oracles/Flag.hs - hadrian/src/Oracles/Flavour.hs - hadrian/src/Oracles/Setting.hs - hadrian/src/Oracles/TestSettings.hs - hadrian/src/Packages.hs - hadrian/src/Rules.hs - hadrian/src/Rules/BinaryDist.hs - hadrian/src/Rules/CabalReinstall.hs - hadrian/src/Rules/Compile.hs - hadrian/src/Rules/Documentation.hs - hadrian/src/Rules/Generate.hs - hadrian/src/Rules/Gmp.hs - hadrian/src/Rules/Libffi.hs - hadrian/src/Rules/Library.hs - hadrian/src/Rules/Program.hs - hadrian/src/Rules/Register.hs - hadrian/src/Rules/Rts.hs - hadrian/src/Rules/Test.hs - hadrian/src/Settings.hs - hadrian/src/Settings/Builders/Cabal.hs - hadrian/src/Settings/Builders/Common.hs - hadrian/src/Settings/Builders/Configure.hs - hadrian/src/Settings/Builders/DeriveConstants.hs - hadrian/src/Settings/Builders/Ghc.hs - hadrian/src/Settings/Builders/Hsc2Hs.hs - hadrian/src/Settings/Builders/RunTest.hs - hadrian/src/Settings/Builders/SplitSections.hs - hadrian/src/Settings/Default.hs - hadrian/src/Settings/Flavours/Benchmark.hs - hadrian/src/Settings/Flavours/Development.hs - hadrian/src/Settings/Flavours/GhcInGhci.hs - hadrian/src/Settings/Flavours/Performance.hs - hadrian/src/Settings/Flavours/Quick.hs - hadrian/src/Settings/Flavours/QuickCross.hs - hadrian/src/Settings/Flavours/Quickest.hs - hadrian/src/Settings/Flavours/Validate.hs - hadrian/src/Settings/Packages.hs - hadrian/src/Settings/Program.hs - hadrian/src/Settings/Warnings.hs - libraries/ghc-heap/GHC/Exts/Heap.hs - m4/fp_find_nm.m4 - m4/fptools_set_platform_vars.m4 - m4/prep_target_file.m4 - testsuite/ghc-config/ghc-config.hs - testsuite/tests/codeGen/should_run/T13825-unit.hs - testsuite/tests/ghc-e/should_fail/all.T - testsuite/tests/unboxedsums/UbxSumUnpackedSize.stdout - testsuite/tests/unboxedsums/UbxSumUnpackedSize.stdout-ws-32 - utils/jsffi/dyld.mjs The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a5449ca95261320942a983e007ec5ea... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/a5449ca95261320942a983e007ec5ea... You're receiving this email because of your account on gitlab.haskell.org.