GitLab

Simon Jakobi pushed to branch wip/sjakobi/T25233 at Glasgow Haskell Compiler / GHC

Commits:

0116eca3

by Simon Jakobi at 2026-08-17T12:27:43+02:00

X86 NCG: handle single-bit literal masks on the left

Constant folding canonicalizes constants to the right of commutable
operations, so these patterns cannot arise from Haskell code — but
hand-written Cmm can reach the NCG unfolded. The variable-index
bit-test equations already match both operand orders.

Assisted-by: Claude Fable 5

235b8823

by Simon Jakobi at 2026-08-17T12:27:54+02:00

X86 NCG: fix overclaim in Note [Bit-test instructions]

The Note claimed that an and/or/xor with an imm32 is "at least as
good" as a bit-test instruction. That overlooks code size: for bit
indices 7..30 the btr/bts/btc encoding is two bytes shorter. Spell
out the actual trade-off (equal latency, better throughput, at worst
two bytes of code size) instead.

Assisted-by: Claude Fable 5

4 changed files:

compiler/GHC/CmmToAsm/X86/CodeGen.hs
+ testsuite/tests/codeGen/should_gen_asm/T25233b.asm
+ testsuite/tests/codeGen/should_gen_asm/T25233b.cmm
testsuite/tests/codeGen/should_gen_asm/all.T

Changes:

compiler/GHC/CmmToAsm/X86/CodeGen.hs

@@ -1480,6 +1480,23 @@ getRegister' platform is32Bit (CmmMachOp (MO_Xor w) [x, CmmLit lit@(CmmInt m _)]
    , not (is32BitLit platform lit)
    = genBitTestImmCode (intFormat w) BTC x i
 +-- Mirrored versions with the literal mask on the left. Constant folding
 +-- canonicalizes constants to the right (see GHC.Cmm.Opt.cmmMachOpFoldM), so
 +-- these only fire on Cmm that reaches the NCG unfolded, e.g. hand-written
 +-- .cmm code.
 +getRegister' platform is32Bit (CmmMachOp (MO_And w) [CmmLit lit@(CmmInt m _), x])
 +  | Just i <- clearBitLit_maybe w m, bitTestOpWidthOK is32Bit w
 +  , not (is32BitLit platform lit)
 +  = genBitTestImmCode (intFormat w) BTR x i
 +getRegister' platform is32Bit (CmmMachOp (MO_Or w) [CmmLit lit@(CmmInt m _), x])
 +  | Just i <- setBitLit_maybe w m, bitTestOpWidthOK is32Bit w
 +  , not (is32BitLit platform lit)
 +  = genBitTestImmCode (intFormat w) BTS x i
 +getRegister' platform is32Bit (CmmMachOp (MO_Xor w) [CmmLit lit@(CmmInt m _), x])
 +  | Just i <- setBitLit_maybe w m, bitTestOpWidthOK is32Bit w
 +  , not (is32BitLit platform lit)
 +  = genBitTestImmCode (intFormat w) BTC x i
++
  getRegister' platform is32Bit (CmmMachOp mop [x, y]) = do -- dyadic MachOps
    sse4_1 <- sse4_1Enabled
    sse4_2 <- sse4_2Enabled
@@ -5938,11 +5955,14 @@ in any case undefined).
  The bit-offset operand of these instructions must be an immediate or a
  register.  When the bit index is a literal, no shift reaches the NCG:
  constant folding has already turned the whole mask into a literal.  If that
 -mask fits in an imm32, an ordinary and/or/xor with an immediate is at least
 -as good; but a W64 mask touching the upper bits, e.g. ~(1 << 40), would have
 -to be moved into a register first.  For such masks we recognise the folded
 -literal itself (exactly one bit clear resp. set) and emit btr/bts/btc with
 -an immediate bit offset.
 +mask fits in an imm32, we keep the ordinary and/or/xor with an immediate:
 +it has the same latency and better throughput (more execution ports) than
 +the bit-test instructions,  and at worst two bytes of extra code size for bit
 +indices 7..30.
 +But a W64 mask touching the upper bits, e.g. ~(1 << 40), would have to be moved
 +into a register first.  For such masks we recognise the folded literal itself
 +(exactly one bit clear resp. set) and emit btr/bts/btc with an immediate
 +bit offset.
  We restrict the pattern to W32 and native-width W64: the instructions do not
  exist at width 8, and sub-word Cmm operations at W8/W16 are rare enough that

testsuite/tests/codeGen/should_gen_asm/T25233b.asm

 +btrq $40,
 +btsq $40,
 +btcq $40,

testsuite/tests/codeGen/should_gen_asm/T25233b.cmm

 +#include "Cmm.h"
++
 +// Single-bit literal masks written on the left of the operator. Constant
 +// folding canonicalizes literals to the right, so these shapes only reach
 +// the NCG from hand-written Cmm like this (#25233).
++
 +clearBit40 (W_ x) {
 +    return ((0xFFFFFEFFFFFFFFFF :: bits64) & x);
 +}
++
 +setBit40 (W_ x) {
 +    return ((0x10000000000 :: bits64) | x);
 +}
++
 +complementBit40 (W_ x) {
 +    return ((0x10000000000 :: bits64) ^ x);
 +}

testsuite/tests/codeGen/should_gen_asm/all.T

@@ -25,6 +25,8 @@ test('avx512-word64-minmax', [unless(arch('x86_64'), skip),
                                when(unregisterised(), skip)], compile_grep_asm, ['hs', True, '-mavx512vl'])
  test('T25233', [unless(arch('x86_64'), skip),
                  when(unregisterised(), skip)], compile_grep_asm, ['hs', True, '-O'])
 +test('T25233b', [unless(arch('x86_64'), skip),
 +                 when(unregisterised(), skip)], compile_grep_asm, ['cmm', True, ''])
  is_aarch64_codegen = [
      unless(arch('aarch64'), skip),
      when(unregisterised(), skip),