Andreas Klebinger pushed to branch wip/andreask/9.10.4-batch1 at Glasgow Haskell Compiler / GHC Commits: b4dc7b1c by Cheng Shao at 2026-10-28T11:42:58+01:00 hadrian: fix GHC.Platform.Host generation for cross stage1 This patch fixes incorrectly GHC.Platform.Host generation logic for cross stage1 in hadrian (#26449). Also adds T26449 test case to witness the fix. Co-authored-by: Codex <codex@openai.com> (cherry picked from commit 8cbe006ad09d5a64e4a3cdf4c91a8b81ff1511be) - - - - - 0eb7b151 by Ben Gamari at 2026-10-28T11:43:25+01:00 gitlab-ci: Make RELEASE_JOB an input Rather than an undocumented variable. (cherry picked from commit f9790ca81deb8b14ff2eabf701aecbcfd6501963) - - - - - a026b215 by Ben Gamari at 2026-10-28T11:43:38+01:00 rts/nonmoving: Fix comment spelling (cherry picked from commit 14281a22eb27498886def8e5d17797c9ce62f3ad) - - - - - b30ecb58 by Ben Gamari at 2026-10-28T11:43:38+01:00 rts/nonmoving: Use atomic operations to update bd->flags (cherry picked from commit bedd38b01d6b113cb3bd10b5d784c16b32efb5bb) - - - - - ce2add9b by Ben Gamari at 2026-10-28T11:43:38+01:00 nonmoving: Use get_itbl instead of explicit loads This is cleaner and also fixes unnecessary (and unsound) use of `volatile`. (cherry picked from commit 215d68414020dc4ed0636508c9eecd9f44f62168) - - - - - 374ca5c4 by Ben Gamari at 2026-10-28T11:43:38+01:00 rts/Scav: Handle WHITEHOLEs in scavenge_one `scavenge_one`, used to scavenge mutable list entries, may encounter `WHITEHOLE`s when the non-moving GC is in use via two paths: 1. when an MVAR is being marked concurrently 2. when the object belongs to a chain of selectors being short-cutted. Fixes #26204. (cherry picked from commit 2c94aa3aa87c14b1ff5c4355c9a90efedd5d10f4) - - - - - 97a0f7a4 by Cheng Shao at 2026-10-28T11:45:58+01:00 testsuite: fix T3586 for non-SSE3 platforms `T3586.hs` contains `-fvia-C -optc-msse3` which I think is a best-effort basis to harvest the C compiler's auto vectorization optimizations via the C backend back when the test was added. The `-fvia-C` part is now a deprecated no-op because GHC can't fall back to the C backend on a non-unregisterised build, and `-optc-msse3` might actually cause the test to fail on non x86/x64 platforms, e.g. recent builds of wasi-sdk would report `wasm32-wasi-clang: error: unsupported option '-msse3' for target 'wasm32-unknown-wasi'`. So this patch cleans up this historical cruft. `-fvia-C` is removed, and `-optc-msse3` is only passed when cpuid contains `pni` (which indicates support of SSE3). (cherry picked from commit 70ee825a516bcf7aac762bfedb4a017d35f8dcf3) - - - - - ba4aac3d by Andreas Klebinger at 2026-10-28T11:53:47+01:00 Add a perf test for #26425 (cherry picked from commit 4038a28b8594ae91c000d5bc7fceed4815eb4826) - - - - - 5a256308 by Andreas Klebinger at 2026-10-28T14:08:55+01:00 OccAnal: Be stricter for better compiler perf. In particular we are now stricter: * When combining usageDetails. * When computing binder info. In combineUsageDetails when combining the underlying adds we compute a new `LocalOcc` for each entry by combining the two existing ones. Rather than wait for those entries to be forced down the road we now force them immediately. Speeding up T26425 by about 10% with little effect on the common case. We also force binders we put into the Core AST everywhere now. Failure to do so risks leaking the occ env used to set the binders OccInfo. For T26425 compiler residency went down by a factor of ~10x. Compile time also improved by a factor of ~1.6. ------------------------- Metric Decrease: T18698a T26425 T9233 ------------------------- (cherry picked from commit f997618e08ce5aad8b7564b8073889b52bc18700) - - - - - 16 changed files: - .gitlab-ci.yml - compiler/GHC/Core/Opt/OccurAnal.hs - compiler/GHC/Types/Unique/FM.hs - compiler/GHC/Types/Var/Env.hs - hadrian/src/Rules/Generate.hs - rts/include/rts/storage/Block.h - rts/sm/NonMoving.c - rts/sm/NonMovingMark.c - rts/sm/Scav.c - testsuite/driver/cpu_features.py - + testsuite/tests/cross/should_run/T26449.hs - + testsuite/tests/cross/should_run/all.T - + testsuite/tests/perf/compiler/T26425.hs - testsuite/tests/perf/compiler/all.T - testsuite/tests/perf/should_run/T3586.hs - testsuite/tests/perf/should_run/all.T Changes: ===================================== .gitlab-ci.yml ===================================== @@ -1,3 +1,12 @@ +spec: + inputs: + release: + default: "no" + description: Release pipeline + options: [ "yes", "no" ] + +--- + variables: GIT_SSL_NO_VERIFY: "1" @@ -25,6 +34,8 @@ variables: # Makes ci.sh isolate CABAL_DIR HERMETIC: "YES" + RELEASE_JOB: $[[ inputs.release ]] + # Reduce XZ compression level for regular jobs (it is bumped to 9 for releases # and nightly jobs). In my experiments I've got the following bindist size in # the given time for each compression level (with the quick flavour): ===================================== compiler/GHC/Core/Opt/OccurAnal.hs ===================================== @@ -9,6 +9,8 @@ -- many /other/ arguments the function has. Inconsistent unboxing is very -- bad for performance, so I increased the limit to allow it to unbox -- consistently. +-- AK: Seems we no longer unbox OccEnv now anyway so it might be redundant. + {- (c) The GRASP/AQUA Project, Glasgow University, 1992-1998 @@ -967,6 +969,12 @@ occAnalBind -> ([CoreBind] -> r -> r) -- How to combine the scope with new binds -> WithUsageDetails r -- Of the whole let(rec) +-- AK: While not allocating any less inlining occAnalBind turns calls to the +-- passed functions into known calls with all the benefits that brings. +-- On a version of T26425 with 6k alternatives this improved compile +-- by 10-20% with -O. +{-# INLINE occAnalBind #-} + occAnalBind env lvl ire (Rec pairs) thing_inside combine = addInScopeList env (map fst pairs) $ \env -> let WUD body_uds body' = thing_inside env @@ -984,7 +992,7 @@ occAnalBind !env lvl ire (NonRec bndr rhs) thing_inside combine = -- Analyse the RHS and /then/ the body let -- Analyse the rhs first, generating rhs_uds !(rhs_uds_s, bndr', rhs') = occAnalNonRecRhs env ire mb_join bndr rhs - rhs_uds = foldr1 orUDs rhs_uds_s -- NB: orUDs. See (W4) of + rhs_uds = foldl1' orUDs rhs_uds_s -- NB: orUDs. See (W4) of -- Note [Occurrence analysis for join points] -- Now analyse the body, adding the join point @@ -1049,6 +1057,7 @@ occAnalNonRecRhs !env imp_rule_edges mb_join bndr rhs -- Match join arity O from mb_join_arity with manifest join arity M as -- returned by of occAnalLamTail. It's totally OK for them to mismatch; -- hence adjust the UDs from the RHS + WUD adj_rhs_uds final_rhs = adjustNonRecRhs mb_join $ occAnalLamTail rhs_env rhs final_bndr_with_rules @@ -2053,6 +2062,18 @@ So The Plan is this: was a loop breaker last time round Hence the is_lb field of NodeScore + +Note [Strictness in the occurrence analyser] +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +By carefully making the occurrence analyser strict in some places, we can +dramatically reduce its memory residency. Among other things we: +* Evaluate the result of `tagLamBinder` and friends, so that the binder (or its + OccInfo) does not retain the entire `UsageDetails`. Also use `strictMap` in `tagLamBinders`. +* In `combineUsageDetailsWith`, the fields of the data constructor are strict, and we use + `strictPlusVarEnv` on the maps that are bound to be needed later on to avoid thunks being + stored in the values. + +These measures reduced residency for test T26425 by a factor of at least 5x. -} {- ********************************************************************* @@ -2187,7 +2208,9 @@ occ_anal_lam_tail env expr@(Lam {}) go env rev_bndrs body = addInScope env rev_bndrs $ \env -> let !(WUD usage body') = occ_anal_lam_tail env body - wrap_lam body bndr = Lam (tagLamBinder usage bndr) body + -- See Note [Strictness in the occurrence analyser] + wrap_lam !body !bndr = let !bndr' = tagLamBinder usage bndr + in Lam bndr' body in WUD (usage `addLamCoVarOccs` rev_bndrs) (foldl' wrap_lam body' rev_bndrs) @@ -2540,7 +2563,8 @@ occAnal env (Case scrut bndr ty alts) let alt_env = addBndrSwap scrut' bndr $ setTailCtxt env -- Kill off OccRhs WUD alts_usage alts' = do_alts alt_env alts - tagged_bndr = tagLamBinder alts_usage bndr + !tagged_bndr = tagLamBinder alts_usage bndr + -- See Note [Strictness in the occurrence analyser] in WUD alts_usage (tagged_bndr, alts') total_usage = markAllNonTail scrut_usage `andUDs` alts_usage @@ -2558,11 +2582,13 @@ occAnal env (Case scrut bndr ty alts) do_alt !env (Alt con bndrs rhs) = addInScopeList env bndrs $ \ env -> let WUD rhs_usage rhs' = occAnal env rhs - tagged_bndrs = tagLamBinders rhs_usage bndrs + !tagged_bndrs = tagLamBinders rhs_usage bndrs + -- See Note [Strictness in the occurrence analyser] in -- See Note [Binders in case alternatives] WUD rhs_usage (Alt con tagged_bndrs rhs') occAnal env (Let bind body) + -- TODO: Would be nice to use a strict version of mkLets here = occAnalBind env NotTopLevel noImpRuleEdges bind (\env -> occAnal env body) mkLets @@ -2622,10 +2648,12 @@ occAnalApp !env (Var fun, args, ticks) | fun `hasKey` runRWKey , [t1, t2, arg] <- args , WUD usage arg' <- adjustNonRecRhs (JoinPoint 1) $ occAnalLamTail env arg - = WUD usage (mkTicks ticks $ mkApps (Var fun) [t1, t2, arg']) + = let app_out = mkTicks ticks $ mkApps (Var fun) [t1, t2, arg'] + in WUD usage app_out occAnalApp env (Var fun_id, args, ticks) - = WUD all_uds (mkTicks ticks app') + = let app_out = mkTicks ticks app' + in WUD all_uds app_out where -- Lots of banged bindings: this is a very heavily bit of code, -- so it pays not to make lots of thunks here, all of which @@ -2670,8 +2698,9 @@ occAnalApp env (Var fun_id, args, ticks) -- See Note [Sources of one-shot information], bullet point A'] occAnalApp env (fun, args, ticks) - = WUD (markAllNonTail (fun_uds `andUDs` args_uds)) - (mkTicks ticks app') + = let app_out = mkTicks ticks app' + in WUD (markAllNonTail (fun_uds `andUDs` args_uds)) app_out + where !(WUD args_uds app') = occAnalArgs env fun' args [] !(WUD fun_uds fun') = occAnal (addAppCtxt env args) fun @@ -3613,8 +3642,8 @@ data WithTailUsageDetails a = WTUD !TailUsageDetails !a ------------------- -- UsageDetails API -andUDs, orUDs - :: UsageDetails -> UsageDetails -> UsageDetails +andUDs:: UsageDetails -> UsageDetails -> UsageDetails +orUDs :: UsageDetails -> UsageDetails -> UsageDetails andUDs = combineUsageDetailsWith andLocalOcc orUDs = combineUsageDetailsWith orLocalOcc @@ -3729,10 +3758,13 @@ combineUsageDetailsWith plus_occ_info | isEmptyVarEnv env1 = uds2 | isEmptyVarEnv env2 = uds1 | otherwise - = UD { ud_env = plusVarEnv_C plus_occ_info env1 env2 - , ud_z_many = plusVarEnv z_many1 z_many2 + -- See Note [Strictness in the occurrence analyser] + -- Using strictPlusVarEnv here speeds up the test T26425 by about 10% by avoiding + -- intermediate thunks. + = UD { ud_env = strictPlusVarEnv_C plus_occ_info env1 env2 + , ud_z_many = strictPlusVarEnv z_many1 z_many2 , ud_z_in_lam = plusVarEnv z_in_lam1 z_in_lam2 - , ud_z_tail = plusVarEnv z_tail1 z_tail2 } + , ud_z_tail = strictPlusVarEnv z_tail1 z_tail2 } lookupLetOccInfo :: UsageDetails -> Id -> OccInfo -- Don't use locally-generated occ_info for exported (visible-elsewhere) @@ -3810,7 +3842,8 @@ tagLamBinders :: UsageDetails -- Of scope -> [Id] -- Binders -> [IdWithOccInfo] -- Tagged binders tagLamBinders usage binders - = map (tagLamBinder usage) binders + -- See Note [Strictness in the occurrence analyser] + = strictMap (tagLamBinder usage) binders tagLamBinder :: UsageDetails -- Of scope -> Id -- Binder @@ -3819,6 +3852,7 @@ tagLamBinder :: UsageDetails -- Of scope -- No-op on TyVars -- A lambda binder never has an unfolding, so no need to look for that tagLamBinder usage bndr + -- See Note [Strictness in the occurrence analyser] = setBinderOcc (markNonTail occ) bndr -- markNonTail: don't try to make an argument into a join point where ===================================== compiler/GHC/Types/Unique/FM.hs ===================================== @@ -51,7 +51,9 @@ module GHC.Types.Unique.FM ( delListFromUFM, delListFromUFM_Directly, plusUFM, + strictPlusUFM, plusUFM_C, + strictPlusUFM_C, plusUFM_CD, plusUFM_CD2, mergeUFM, @@ -249,16 +251,24 @@ delListFromUFM_Directly = foldl' delFromUFM_Directly delFromUFM_Directly :: UniqFM key elt -> Unique -> UniqFM key elt delFromUFM_Directly (UFM m) u = UFM (M.delete (getKey u) m) --- Bindings in right argument shadow those in the left +-- | Bindings in right argument shadow those in the left. +-- +-- Unlike containers this union is right-biased for historic reasons. plusUFM :: UniqFM key elt -> UniqFM key elt -> UniqFM key elt --- M.union is left-biased, plusUFM should be right-biased. plusUFM (UFM x) (UFM y) = UFM (M.union y x) -- Note (M.union y x), with arguments flipped -- M.union is left-biased, plusUFM should be right-biased. +-- | Right biased +strictPlusUFM :: UniqFM key elt -> UniqFM key elt -> UniqFM key elt +strictPlusUFM (UFM x) (UFM y) = UFM (MS.union y x) + plusUFM_C :: (elt -> elt -> elt) -> UniqFM key elt -> UniqFM key elt -> UniqFM key elt plusUFM_C f (UFM x) (UFM y) = UFM (M.unionWith f x y) +strictPlusUFM_C :: (elt -> elt -> elt) -> UniqFM key elt -> UniqFM key elt -> UniqFM key elt +strictPlusUFM_C f (UFM x) (UFM y) = UFM (MS.unionWith f x y) + -- | `plusUFM_CD f m1 d1 m2 d2` merges the maps using `f` as the -- combinding function and `d1` resp. `d2` as the default value if -- there is no entry in `m1` reps. `m2`. The domain is the union of ===================================== compiler/GHC/Types/Var/Env.hs ===================================== @@ -12,7 +12,8 @@ module GHC.Types.Var.Env ( elemVarEnv, disjointVarEnv, anyVarEnv, extendVarEnv, extendVarEnv_C, extendVarEnv_Acc, extendVarEnvList, - plusVarEnv, plusVarEnv_C, plusVarEnv_CD, plusMaybeVarEnv_C, + strictPlusVarEnv, plusVarEnv, plusVarEnv_C, strictPlusVarEnv_C, + plusVarEnv_CD, plusMaybeVarEnv_C, plusVarEnvList, alterVarEnv, delVarEnvList, delVarEnv, minusVarEnv, @@ -510,6 +511,7 @@ extendVarEnv :: VarEnv a -> Var -> a -> VarEnv a extendVarEnv_C :: (a->a->a) -> VarEnv a -> Var -> a -> VarEnv a extendVarEnv_Acc :: (a->b->b) -> (a->b) -> VarEnv b -> Var -> a -> VarEnv b plusVarEnv :: VarEnv a -> VarEnv a -> VarEnv a +strictPlusVarEnv :: VarEnv a -> VarEnv a -> VarEnv a plusVarEnvList :: [VarEnv a] -> VarEnv a extendVarEnvList :: VarEnv a -> [(Var, a)] -> VarEnv a varEnvDomain :: VarEnv elt -> UnVarSet @@ -521,6 +523,7 @@ delVarEnvList :: VarEnv a -> [Var] -> VarEnv a delVarEnv :: VarEnv a -> Var -> VarEnv a minusVarEnv :: VarEnv a -> VarEnv b -> VarEnv a plusVarEnv_C :: (a -> a -> a) -> VarEnv a -> VarEnv a -> VarEnv a +strictPlusVarEnv_C :: (a -> a -> a) -> VarEnv a -> VarEnv a -> VarEnv a plusVarEnv_CD :: (a -> a -> a) -> VarEnv a -> a -> VarEnv a -> a -> VarEnv a plusMaybeVarEnv_C :: (a -> a -> Maybe a) -> VarEnv a -> VarEnv a -> VarEnv a mapVarEnv :: (a -> b) -> VarEnv a -> VarEnv b @@ -547,12 +550,14 @@ extendVarEnv_C = addToUFM_C extendVarEnv_Acc = addToUFM_Acc extendVarEnvList = addListToUFM plusVarEnv_C = plusUFM_C +strictPlusVarEnv_C = strictPlusUFM_C plusVarEnv_CD = plusUFM_CD plusMaybeVarEnv_C = plusMaybeUFM_C delVarEnvList = delListFromUFM delVarEnv = delFromUFM minusVarEnv = minusUFM plusVarEnv = plusUFM +strictPlusVarEnv = strictPlusUFM plusVarEnvList = plusUFMList -- lookupVarEnv is very hot (in part due to being called by substTyVar), -- if it's not inlined than the mere allocation of the Just constructor causes ===================================== hadrian/src/Rules/Generate.hs ===================================== @@ -570,8 +570,12 @@ generateVersionHs = do generatePlatformHostHs :: Expr String generatePlatformHostHs = do trackGenerateHs - cHostPlatformArch <- queryHost (archOS_arch . tgtArchOs) - cHostPlatformOS <- queryHost (archOS_OS . tgtArchOs) + stage <- getStage + let chooseHostQuery = case stage of + Stage0 {} -> queryHost + _ -> queryTarget + cHostPlatformArch <- chooseHostQuery (archOS_arch . tgtArchOs) + cHostPlatformOS <- chooseHostQuery (archOS_OS . tgtArchOs) return $ unlines [ "module GHC.Platform.Host where" , "" ===================================== rts/include/rts/storage/Block.h ===================================== @@ -300,6 +300,28 @@ dbl_link_replace(bdescr *new_, bdescr *old, bdescr **list) } } +INLINE_HEADER void +block_set_flag(bdescr *bd, uint16_t flag) +{ + // TODO: This ordering is stronger than necessary for many users (e.g. + // setting flags). + __atomic_or_fetch(&bd->flags, flag, __ATOMIC_SEQ_CST); +} + +INLINE_HEADER void +block_clear_flag(bdescr *bd, uint16_t flag) +{ + // TODO: This ordering is stronger than necessary for many users (e.g. + // setting flags). + __atomic_and_fetch(&bd->flags, ~flag, __ATOMIC_SEQ_CST); +} + +INLINE_HEADER uint16_t +block_get_flags(bdescr *bd) +{ + return RELAXED_LOAD(&bd->flags); +} + /* Initialisation ---------------------------------------------------------- */ extern void initBlockAllocator(void); ===================================== rts/sm/NonMoving.c ===================================== @@ -550,7 +550,7 @@ static void nonmovingBumpEpoch(void) { * arbitrary allocator sizes, we need to do some precomputation and make * use of the integer division by constants optimisation. * - * We currenlty try to balance these considerations by adopting the following scheme. + * We currently try to balance these considerations by adopting the following scheme. * We have nonmoving_alloca_dense_cnt "dense" allocators starting with size * NONMOVING_ALLOCA0, and incrementing by NONMOVING_ALLOCA_DENSE_INCREMENT. * These service the vast majority of allocations. ===================================== rts/sm/NonMovingMark.c ===================================== @@ -606,7 +606,7 @@ static bool check_in_nonmoving_heap(StgClosure *p) { if (HEAP_ALLOCED_GC(p)) { // This works for both large and small objects: - return Bdescr((P_)p)->flags & BF_NONMOVING; + return block_get_flags(Bdescr((P_)p)) & BF_NONMOVING; } else { return true; // a static object } @@ -619,7 +619,7 @@ inline void updateRemembSetPushThunk(Capability *cap, StgThunk *thunk) { const StgInfoTable *info; do { - info = *(StgInfoTable* volatile*) &thunk->header.info; + info = (StgInfoTable*) RELAXED_LOAD(&thunk->header.info); } while (info == &stg_WHITEHOLE_info); const StgThunkInfoTable *thunk_info = THUNK_INFO_PTR_TO_STRUCT(info); @@ -722,13 +722,14 @@ STATIC_INLINE bool needs_upd_rem_set_mark(StgClosure *p) { // TODO: Deduplicate with mark_closure bdescr *bd = Bdescr((StgPtr) p); + uint16_t flags = block_get_flags(bd); if (bd->gen != oldest_gen) { return false; - } else if (bd->flags & BF_LARGE) { - if (! (bd->flags & BF_NONMOVING_SWEEPING)) { + } else if (flags & BF_LARGE) { + if (! (flags & BF_NONMOVING_SWEEPING)) { return false; } else { - return ! (bd->flags & BF_MARKED); + return ! (flags & BF_MARKED); } } else { struct NonmovingSegment *seg = nonmovingGetSegment((StgPtr) p); @@ -740,8 +741,8 @@ STATIC_INLINE bool needs_upd_rem_set_mark(StgClosure *p) static void finish_upd_rem_set_mark_large(bdescr* bd) { // Someone else may have already marked it. ACQUIRE_LOCK(&nonmoving_large_objects_mutex); - if (! (bd->flags & BF_MARKED)) { - bd->flags |= BF_MARKED; + if (! (block_get_flags(bd) & BF_MARKED)) { + block_set_flag(bd, BF_MARKED); dbl_link_remove(bd, &nonmoving_large_objects); dbl_link_onto(bd, &nonmoving_marked_large_objects); n_nonmoving_large_blocks -= bd->blocks; @@ -754,7 +755,7 @@ static void finish_upd_rem_set_mark_large(bdescr* bd) { STATIC_INLINE void finish_upd_rem_set_mark(StgClosure *p) { bdescr *bd = Bdescr((StgPtr) p); - if (bd->flags & BF_LARGE) { + if (block_get_flags(bd) & BF_LARGE) { // This function is extracted so that this function can be inline finish_upd_rem_set_mark_large(bd); } else { @@ -1342,7 +1343,7 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin) goto done; case WHITEHOLE: - while (*(StgInfoTable* volatile*) &p->header.info == &stg_WHITEHOLE_info) + while (RELAXED_LOAD(&p->header.info) == &stg_WHITEHOLE_info) #if defined(PARALLEL_GC) busy_wait_nop() #endif @@ -1376,35 +1377,36 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin) // N.B. only the first block of a compact region is guaranteed to carry // BF_NONMOVING; consequently we must separately check for BF_COMPACT. - if (bd->flags & (BF_COMPACT | BF_NONMOVING)) { + const uint16_t flags = block_get_flags(bd); + if (flags & (BF_COMPACT | BF_NONMOVING)) { - if (bd->flags & BF_COMPACT) { + if (flags & BF_COMPACT) { StgCompactNFData *str = objectGetCompact((StgClosure*)p); bd = Bdescr((P_)str); - if (! (bd->flags & BF_NONMOVING_SWEEPING)) { + if (! (flags & BF_NONMOVING_SWEEPING)) { // Not in the snapshot return; } - if (! (bd->flags & BF_MARKED)) { + if (! (flags & BF_MARKED)) { dbl_link_remove(bd, &nonmoving_compact_objects); dbl_link_onto(bd, &nonmoving_marked_compact_objects); StgWord blocks = str->totalW / BLOCK_SIZE_W; n_nonmoving_compact_blocks -= blocks; n_nonmoving_marked_compact_blocks += blocks; - bd->flags |= BF_MARKED; + block_set_flag(bd, BF_MARKED); } // N.B. the object being marked is in a compact region so by // definition there is no need to do any tracing here. goto done; - } else if (bd->flags & BF_LARGE) { - if (! (bd->flags & BF_NONMOVING_SWEEPING)) { + } else if (flags & BF_LARGE) { + if (! (flags & BF_NONMOVING_SWEEPING)) { // Not in the snapshot goto done; } - if (bd->flags & BF_MARKED) { + if (flags & BF_MARKED) { goto done; } } else { @@ -1712,7 +1714,7 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin) break; case WHITEHOLE: - while (*(StgInfoTable* volatile*) &p->header.info == &stg_WHITEHOLE_info); + while ((StgInfoTable *) RELAXED_LOAD(&p->header.info) == &stg_WHITEHOLE_info); goto try_again; case COMPACT_NFDATA: @@ -1736,24 +1738,25 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin) * the object's pointers since in the case of marking stacks there may be a * mutator waiting for us to finish so it can start execution. */ - if (bd->flags & BF_LARGE) { + uint16_t bd_flags = block_get_flags(bd); + if (bd_flags & BF_LARGE) { /* Marking a large object isn't idempotent since we move it to * nonmoving_marked_large_objects; to ensure that we don't repeatedly * mark a large object, we only set BF_MARKED on large objects in the * nonmoving heap while holding nonmoving_large_objects_mutex */ ACQUIRE_LOCK(&nonmoving_large_objects_mutex); - if (! (bd->flags & BF_MARKED)) { + if (! (bd_flags & BF_MARKED)) { // Remove the object from nonmoving_large_objects and link it to // nonmoving_marked_large_objects dbl_link_remove(bd, &nonmoving_large_objects); dbl_link_onto(bd, &nonmoving_marked_large_objects); n_nonmoving_large_blocks -= bd->blocks; n_nonmoving_marked_large_blocks += bd->blocks; - bd->flags |= BF_MARKED; + block_set_flag(bd, BF_MARKED); } RELEASE_LOCK(&nonmoving_large_objects_mutex); - } else if (bd->flags & BF_NONMOVING) { + } else if (bd_flags & BF_NONMOVING) { // TODO: Kill repetition struct NonmovingSegment *seg = nonmovingGetSegment((StgPtr) p); nonmoving_block_idx block_idx = nonmovingGetBlockIdx((StgPtr) p); @@ -1768,7 +1771,7 @@ mark_closure (MarkQueue *queue, const StgClosure *p0, StgClosure **origin) } done: - if (origin != NULL && (!HEAP_ALLOCED(p) || bd->flags & BF_NONMOVING)) { + if (origin != NULL && (!HEAP_ALLOCED(p) || block_get_flags(bd) & BF_NONMOVING)) { if (UNTAG_CLOSURE((StgClosure*)p0) != p && *origin == p0) { if (cas((StgVolatilePtr)origin, (StgWord)p0, (StgWord)TAG_CLOSURE(tag, p)) == (StgWord)p0) { // debugBelch("Thunk optimization successful\n"); @@ -1865,19 +1868,20 @@ bool nonmovingIsAlive (StgClosure *p) } bdescr *bd = Bdescr((P_)p); + uint16_t bd_flags = block_get_flags(bd); // All non-static objects in the non-moving heap should be marked as // BF_NONMOVING - ASSERT(bd->flags & BF_NONMOVING); + ASSERT(bd_flags & BF_NONMOVING); - if (bd->flags & (BF_COMPACT | BF_LARGE)) { - if (bd->flags & BF_COMPACT) { + if (bd_flags & (BF_COMPACT | BF_LARGE)) { + if (bd_flags & BF_COMPACT) { StgCompactNFData *str = objectGetCompact((StgClosure*)p); bd = Bdescr((P_)str); } - return (bd->flags & BF_NONMOVING_SWEEPING) == 0 + return (bd_flags & BF_NONMOVING_SWEEPING) == 0 // the large object wasn't in the snapshot and therefore wasn't marked - || (bd->flags & BF_MARKED) != 0; + || (bd_flags & BF_MARKED) != 0; // The object was marked } else { struct NonmovingSegment *seg = nonmovingGetSegment((StgPtr) p); @@ -1931,8 +1935,8 @@ static bool nonmovingIsNowAlive (StgClosure *p) } bdescr *bd = Bdescr((P_)p); + const uint16_t flags = block_get_flags(bd); - const uint16_t flags = bd->flags; if (flags & BF_LARGE) { if (flags & BF_PINNED && !(flags & BF_NONMOVING)) { // In this case we have a pinned object living in a non-full @@ -1942,15 +1946,15 @@ static bool nonmovingIsNowAlive (StgClosure *p) return true; } - ASSERT(bd->flags & BF_NONMOVING); - return (bd->flags & BF_NONMOVING_SWEEPING) == 0 + ASSERT(flags & BF_NONMOVING); + return (flags & BF_NONMOVING_SWEEPING) == 0 // the large object wasn't in the snapshot and therefore wasn't marked - || (bd->flags & BF_MARKED) != 0; + || (flags & BF_MARKED) != 0; // The object was marked } else { // All non-static objects in the non-moving heap should be marked as // BF_NONMOVING. - ASSERT(bd->flags & BF_NONMOVING); + ASSERT(flags & BF_NONMOVING); struct NonmovingSegment *seg = nonmovingGetSegment((StgPtr) p); StgClosure *snapshot_loc = @@ -2013,7 +2017,8 @@ bool nonmovingTidyWeaks (struct MarkQueue_ *queue) // See Note [Weak pointer processing and the non-moving GC] in // MarkWeak.c - bool key_in_nonmoving = HEAP_ALLOCED_GC(w->key) && Bdescr((StgPtr) w->key)->flags & BF_NONMOVING; + bdescr *key_bd = Bdescr((StgPtr) w->key); + bool key_in_nonmoving = HEAP_ALLOCED_GC(w->key) && block_get_flags(key_bd) & BF_NONMOVING; if (!key_in_nonmoving || nonmovingIsNowAlive(w->key)) { nonmovingMarkLiveWeak(queue, w); did_work = true; ===================================== rts/sm/Scav.c ===================================== @@ -1269,6 +1269,7 @@ scavenge_one(StgPtr p) bool no_luck; bool saved_eager_promotion; +try_again: saved_eager_promotion = gct->eager_promotion; ASSERT(LOOKS_LIKE_CLOSURE_PTR(p)); @@ -1610,6 +1611,13 @@ scavenge_one(StgPtr p) scavenge_continuation((StgContinuation *)p); break; + case WHITEHOLE: + // This may happen when the nonmoving GC is in use via two paths: + // 1. when an MVAR is being marked concurrently + // 2. when the object belongs to a chain of selectors being short-cutted. + while (get_itbl((StgClosure*) p) == &stg_WHITEHOLE_info); + goto try_again; + default: barf("scavenge_one: strange object %d", (int)(info->type)); } ===================================== testsuite/driver/cpu_features.py ===================================== @@ -8,8 +8,8 @@ SUPPORTED_CPU_FEATURES = { # These aren't comprehensive; they are only CPU features that we care about # x86: - 'sse', 'sse2', 'sse3', 'ssse3', 'sse4_1', 'sse4_2', - 'avx1', 'avx2', + 'sse', 'sse2', 'sse3', 'pni', 'ssse3', 'sse4_1', 'sse4_2', + 'avx', 'avx2', 'avx512f', 'fma', 'popcnt', 'bmi1', 'bmi2' } ===================================== testsuite/tests/cross/should_run/T26449.hs ===================================== @@ -0,0 +1,16 @@ +import Control.Monad +import GHC.Platform.ArchOS +import GHC.Platform.Host +import System.Info + +main :: IO () +main = + when ((arch, os) /= (arch', os')) $ + fail $ + "System.Info says host platform is " + <> show (arch, os) + <> " but GHC.Platform.Host says " + <> show (arch', os') + where + (arch', os') = + (stringEncodeArch hostPlatformArch, stringEncodeOS hostPlatformOS) ===================================== testsuite/tests/cross/should_run/all.T ===================================== @@ -0,0 +1 @@ +test('T26449', [], compile_and_run, ['']) ===================================== testsuite/tests/perf/compiler/T26425.hs ===================================== @@ -0,0 +1,514 @@ +{-# LANGUAGE OverloadedStrings #-} + +module Reproducer (strToInt) where + +import qualified Data.Text as T + +{- This program results in a nested chain of join points and cases which tests + primarily OccAnal and Unfolding performance. +-} + +strToInt :: T.Text -> Maybe Int +strToInt txt = case txt of + "0" -> Just 0 + "1" -> Just 1 + "2" -> Just 2 + "3" -> Just 3 + "4" -> Just 4 + "5" -> Just 5 + "6" -> Just 6 + "7" -> Just 7 + "8" -> Just 8 + "9" -> Just 9 + "10" -> Just 10 + "11" -> Just 11 + "12" -> Just 12 + "13" -> Just 13 + "14" -> Just 14 + "15" -> Just 15 + "16" -> Just 16 + "17" -> Just 17 + "18" -> Just 18 + "19" -> Just 19 + "20" -> Just 20 + "21" -> Just 21 + "22" -> Just 22 + "23" -> Just 23 + "24" -> Just 24 + "25" -> Just 25 + "26" -> Just 26 + "27" -> Just 27 + "28" -> Just 28 + "29" -> Just 29 + "30" -> Just 30 + "31" -> Just 31 + "32" -> Just 32 + "33" -> Just 33 + "34" -> Just 34 + "35" -> Just 35 + "36" -> Just 36 + "37" -> Just 37 + "38" -> Just 38 + "39" -> Just 39 + "40" -> Just 40 + "41" -> Just 41 + "42" -> Just 42 + "43" -> Just 43 + "44" -> Just 44 + "45" -> Just 45 + "46" -> Just 46 + "47" -> Just 47 + "48" -> Just 48 + "49" -> Just 49 + "50" -> Just 50 + "51" -> Just 51 + "52" -> Just 52 + "53" -> Just 53 + "54" -> Just 54 + "55" -> Just 55 + "56" -> Just 56 + "57" -> Just 57 + "58" -> Just 58 + "59" -> Just 59 + "60" -> Just 60 + "61" -> Just 61 + "62" -> Just 62 + "63" -> Just 63 + "64" -> Just 64 + "65" -> Just 65 + "66" -> Just 66 + "67" -> Just 67 + "68" -> Just 68 + "69" -> Just 69 + "70" -> Just 70 + "71" -> Just 71 + "72" -> Just 72 + "73" -> Just 73 + "74" -> Just 74 + "75" -> Just 75 + "76" -> Just 76 + "77" -> Just 77 + "78" -> Just 78 + "79" -> Just 79 + "80" -> Just 80 + "81" -> Just 81 + "82" -> Just 82 + "83" -> Just 83 + "84" -> Just 84 + "85" -> Just 85 + "86" -> Just 86 + "87" -> Just 87 + "88" -> Just 88 + "89" -> Just 89 + "90" -> Just 90 + "91" -> Just 91 + "92" -> Just 92 + "93" -> Just 93 + "94" -> Just 94 + "95" -> Just 95 + "96" -> Just 96 + "97" -> Just 97 + "98" -> Just 98 + "99" -> Just 99 + "100" -> Just 100 + "101" -> Just 101 + "102" -> Just 102 + "103" -> Just 103 + "104" -> Just 104 + "105" -> Just 105 + "106" -> Just 106 + "107" -> Just 107 + "108" -> Just 108 + "109" -> Just 109 + "110" -> Just 110 + "111" -> Just 111 + "112" -> Just 112 + "113" -> Just 113 + "114" -> Just 114 + "115" -> Just 115 + "116" -> Just 116 + "117" -> Just 117 + "118" -> Just 118 + "119" -> Just 119 + "120" -> Just 120 + "121" -> Just 121 + "122" -> Just 122 + "123" -> Just 123 + "124" -> Just 124 + "125" -> Just 125 + "126" -> Just 126 + "127" -> Just 127 + "128" -> Just 128 + "129" -> Just 129 + "130" -> Just 130 + "131" -> Just 131 + "132" -> Just 132 + "133" -> Just 133 + "134" -> Just 134 + "135" -> Just 135 + "136" -> Just 136 + "137" -> Just 137 + "138" -> Just 138 + "139" -> Just 139 + "140" -> Just 140 + "141" -> Just 141 + "142" -> Just 142 + "143" -> Just 143 + "144" -> Just 144 + "145" -> Just 145 + "146" -> Just 146 + "147" -> Just 147 + "148" -> Just 148 + "149" -> Just 149 + "150" -> Just 150 + "151" -> Just 151 + "152" -> Just 152 + "153" -> Just 153 + "154" -> Just 154 + "155" -> Just 155 + "156" -> Just 156 + "157" -> Just 157 + "158" -> Just 158 + "159" -> Just 159 + "160" -> Just 160 + "161" -> Just 161 + "162" -> Just 162 + "163" -> Just 163 + "164" -> Just 164 + "165" -> Just 165 + "166" -> Just 166 + "167" -> Just 167 + "168" -> Just 168 + "169" -> Just 169 + "170" -> Just 170 + "171" -> Just 171 + "172" -> Just 172 + "173" -> Just 173 + "174" -> Just 174 + "175" -> Just 175 + "176" -> Just 176 + "177" -> Just 177 + "178" -> Just 178 + "179" -> Just 179 + "180" -> Just 180 + "181" -> Just 181 + "182" -> Just 182 + "183" -> Just 183 + "184" -> Just 184 + "185" -> Just 185 + "186" -> Just 186 + "187" -> Just 187 + "188" -> Just 188 + "189" -> Just 189 + "190" -> Just 190 + "191" -> Just 191 + "192" -> Just 192 + "193" -> Just 193 + "194" -> Just 194 + "195" -> Just 195 + "196" -> Just 196 + "197" -> Just 197 + "198" -> Just 198 + "199" -> Just 199 + "200" -> Just 200 + "201" -> Just 201 + "202" -> Just 202 + "203" -> Just 203 + "204" -> Just 204 + "205" -> Just 205 + "206" -> Just 206 + "207" -> Just 207 + "208" -> Just 208 + "209" -> Just 209 + "210" -> Just 210 + "211" -> Just 211 + "212" -> Just 212 + "213" -> Just 213 + "214" -> Just 214 + "215" -> Just 215 + "216" -> Just 216 + "217" -> Just 217 + "218" -> Just 218 + "219" -> Just 219 + "220" -> Just 220 + "221" -> Just 221 + "222" -> Just 222 + "223" -> Just 223 + "224" -> Just 224 + "225" -> Just 225 + "226" -> Just 226 + "227" -> Just 227 + "228" -> Just 228 + "229" -> Just 229 + "230" -> Just 230 + "231" -> Just 231 + "232" -> Just 232 + "233" -> Just 233 + "234" -> Just 234 + "235" -> Just 235 + "236" -> Just 236 + "237" -> Just 237 + "238" -> Just 238 + "239" -> Just 239 + "240" -> Just 240 + "241" -> Just 241 + "242" -> Just 242 + "243" -> Just 243 + "244" -> Just 244 + "245" -> Just 245 + "246" -> Just 246 + "247" -> Just 247 + "248" -> Just 248 + "249" -> Just 249 + "250" -> Just 250 + "251" -> Just 251 + "252" -> Just 252 + "253" -> Just 253 + "254" -> Just 254 + "255" -> Just 255 + "256" -> Just 256 + "257" -> Just 257 + "258" -> Just 258 + "259" -> Just 259 + "260" -> Just 260 + "261" -> Just 261 + "262" -> Just 262 + "263" -> Just 263 + "264" -> Just 264 + "265" -> Just 265 + "266" -> Just 266 + "267" -> Just 267 + "268" -> Just 268 + "269" -> Just 269 + "270" -> Just 270 + "271" -> Just 271 + "272" -> Just 272 + "273" -> Just 273 + "274" -> Just 274 + "275" -> Just 275 + "276" -> Just 276 + "277" -> Just 277 + "278" -> Just 278 + "279" -> Just 279 + "280" -> Just 280 + "281" -> Just 281 + "282" -> Just 282 + "283" -> Just 283 + "284" -> Just 284 + "285" -> Just 285 + "286" -> Just 286 + "287" -> Just 287 + "288" -> Just 288 + "289" -> Just 289 + "290" -> Just 290 + "291" -> Just 291 + "292" -> Just 292 + "293" -> Just 293 + "294" -> Just 294 + "295" -> Just 295 + "296" -> Just 296 + "297" -> Just 297 + "298" -> Just 298 + "299" -> Just 299 + "300" -> Just 300 + "301" -> Just 301 + "302" -> Just 302 + "303" -> Just 303 + "304" -> Just 304 + "305" -> Just 305 + "306" -> Just 306 + "307" -> Just 307 + "308" -> Just 308 + "309" -> Just 309 + "310" -> Just 310 + "311" -> Just 311 + "312" -> Just 312 + "313" -> Just 313 + "314" -> Just 314 + "315" -> Just 315 + "316" -> Just 316 + "317" -> Just 317 + "318" -> Just 318 + "319" -> Just 319 + "320" -> Just 320 + "321" -> Just 321 + "322" -> Just 322 + "323" -> Just 323 + "324" -> Just 324 + "325" -> Just 325 + "326" -> Just 326 + "327" -> Just 327 + "328" -> Just 328 + "329" -> Just 329 + "330" -> Just 330 + "331" -> Just 331 + "332" -> Just 332 + "333" -> Just 333 + "334" -> Just 334 + "335" -> Just 335 + "336" -> Just 336 + "337" -> Just 337 + "338" -> Just 338 + "339" -> Just 339 + "340" -> Just 340 + "341" -> Just 341 + "342" -> Just 342 + "343" -> Just 343 + "344" -> Just 344 + "345" -> Just 345 + "346" -> Just 346 + "347" -> Just 347 + "348" -> Just 348 + "349" -> Just 349 + "350" -> Just 350 + "351" -> Just 351 + "352" -> Just 352 + "353" -> Just 353 + "354" -> Just 354 + "355" -> Just 355 + "356" -> Just 356 + "357" -> Just 357 + "358" -> Just 358 + "359" -> Just 359 + "360" -> Just 360 + "361" -> Just 361 + "362" -> Just 362 + "363" -> Just 363 + "364" -> Just 364 + "365" -> Just 365 + "366" -> Just 366 + "367" -> Just 367 + "368" -> Just 368 + "369" -> Just 369 + "370" -> Just 370 + "371" -> Just 371 + "372" -> Just 372 + "373" -> Just 373 + "374" -> Just 374 + "375" -> Just 375 + "376" -> Just 376 + "377" -> Just 377 + "378" -> Just 378 + "379" -> Just 379 + "380" -> Just 380 + "381" -> Just 381 + "382" -> Just 382 + "383" -> Just 383 + "384" -> Just 384 + "385" -> Just 385 + "386" -> Just 386 + "387" -> Just 387 + "388" -> Just 388 + "389" -> Just 389 + "390" -> Just 390 + "391" -> Just 391 + "392" -> Just 392 + "393" -> Just 393 + "394" -> Just 394 + "395" -> Just 395 + "396" -> Just 396 + "397" -> Just 397 + "398" -> Just 398 + "399" -> Just 399 + "400" -> Just 400 + "401" -> Just 401 + "402" -> Just 402 + "403" -> Just 403 + "404" -> Just 404 + "405" -> Just 405 + "406" -> Just 406 + "407" -> Just 407 + "408" -> Just 408 + "409" -> Just 409 + "410" -> Just 410 + "411" -> Just 411 + "412" -> Just 412 + "413" -> Just 413 + "414" -> Just 414 + "415" -> Just 415 + "416" -> Just 416 + "417" -> Just 417 + "418" -> Just 418 + "419" -> Just 419 + "420" -> Just 420 + "421" -> Just 421 + "422" -> Just 422 + "423" -> Just 423 + "424" -> Just 424 + "425" -> Just 425 + "426" -> Just 426 + "427" -> Just 427 + "428" -> Just 428 + "429" -> Just 429 + "430" -> Just 430 + "431" -> Just 431 + "432" -> Just 432 + "433" -> Just 433 + "434" -> Just 434 + "435" -> Just 435 + "436" -> Just 436 + "437" -> Just 437 + "438" -> Just 438 + "439" -> Just 439 + "440" -> Just 440 + "441" -> Just 441 + "442" -> Just 442 + "443" -> Just 443 + "444" -> Just 444 + "445" -> Just 445 + "446" -> Just 446 + "447" -> Just 447 + "448" -> Just 448 + "449" -> Just 449 + "450" -> Just 450 + "451" -> Just 451 + "452" -> Just 452 + "453" -> Just 453 + "454" -> Just 454 + "455" -> Just 455 + "456" -> Just 456 + "457" -> Just 457 + "458" -> Just 458 + "459" -> Just 459 + "460" -> Just 460 + "461" -> Just 461 + "462" -> Just 462 + "463" -> Just 463 + "464" -> Just 464 + "465" -> Just 465 + "466" -> Just 466 + "467" -> Just 467 + "468" -> Just 468 + "469" -> Just 469 + "470" -> Just 470 + "471" -> Just 471 + "472" -> Just 472 + "473" -> Just 473 + "474" -> Just 474 + "475" -> Just 475 + "476" -> Just 476 + "477" -> Just 477 + "478" -> Just 478 + "479" -> Just 479 + "480" -> Just 480 + "481" -> Just 481 + "482" -> Just 482 + "483" -> Just 483 + "484" -> Just 484 + "485" -> Just 485 + "486" -> Just 486 + "487" -> Just 487 + "488" -> Just 488 + "489" -> Just 489 + "490" -> Just 490 + "491" -> Just 491 + "492" -> Just 492 + "493" -> Just 493 + "494" -> Just 494 + "495" -> Just 495 + "496" -> Just 496 + "497" -> Just 497 + "498" -> Just 498 + "499" -> Just 499 + "500" -> Just 500 + _ -> Nothing ===================================== testsuite/tests/perf/compiler/all.T ===================================== @@ -716,3 +716,8 @@ test ('LookupFusion', test('T24471', [req_th, collect_compiler_stats('all', 5)], multimod_compile, ['T24471', '-v0 -O']) + +test ('T26425', + [ collect_compiler_stats('all',5) ], + compile, + ['-O']) \ No newline at end of file ===================================== testsuite/tests/perf/should_run/T3586.hs ===================================== @@ -1,5 +1,5 @@ {-# LANGUAGE BangPatterns #-} -{-# OPTIONS -fvia-C -optc-O3 -fexcess-precision -optc-msse3 #-} +{-# OPTIONS -optc-O3 -fexcess-precision #-} import Control.Monad.ST import Data.Array.ST ===================================== testsuite/tests/perf/should_run/all.T ===================================== @@ -43,6 +43,7 @@ test('T3586', [collect_runtime_residency(2), collect_stats('bytes allocated', 5), only_ways(['normal']), + when(have_cpu_feature('pni') or have_cpu_feature('sse3'), extra_hc_opts('-optc-msse3')), ], compile_and_run, ['-O']) View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/e03f58bd5cbf944049017d8dba8aab8... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/e03f58bd5cbf944049017d8dba8aab8... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help