Cheng Shao pushed to branch wip/terrorjack/asan at Glasgow Haskell Compiler / GHC Commits: 91edd292 by Wolfgang Jeltsch at 2025-12-19T03:18:19-05:00 Remove unused known-key and name variables for generics This removes the known-key and corresponding name variables for `K1`, `M1`, `R`, `D`, `C`, `S`, and `URec` from `GHC.Generics`, as they are apparently nowhere used in GHC’s source code. - - - - - 73ee7e38 by Wolfgang Jeltsch at 2025-12-19T03:19:02-05:00 Remove unused known keys and names for generics classes This removes the known-key and corresponding name variables for `Datatype`, `Constructor`, and `Selector` from `GHC.Generics`, as they are apparently nowhere used in GHC’s source code. - - - - - f69c5f14 by Cheng Shao at 2025-12-19T03:19:45-05:00 wasm: fix handling of ByteArray#/MutableByteArray# arguments in JSFFI imports This patch fixes the handling of ByteArray#/MutableByteArray# arguments in JSFFI imports, see the amended note and manual for explanation. Also adds a test to witness the fix. Co-authored-by: Codex - - - - - 224446a2 by Cheng Shao at 2025-12-20T07:49:54-05:00 rts: workaround -Werror=maybe-uninitialized false positives In some cases gcc might report -Werror=maybe-uninitialized that we know are false positives, but need to workaround it to make validate builds with -Werror pass. - - - - - 251ec087 by Cheng Shao at 2025-12-20T07:49:54-05:00 hadrian: use -Og as C/C++ optimization level when debugging This commit enables -Og as optimization level when compiling the debug ways of rts. According to gcc documentation (https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-Og), -Og is a better choice than -O0 for producing debuggable code. It's also supported by clang as well, so it makes sense to use it as a default for debugging. Also add missing -g3 flag to C++ compilation flags in +debug_info flavour transformer. - - - - - fb586c67 by Cheng Shao at 2025-12-20T07:50:36-05:00 compiler: replace DList with OrdList This patch removes `DList` logic from the compiler and replaces it with `OrdList` which also supports O(1) concatenation and should be more memory efficient than the church-encoded `DList`. - - - - - 8149c987 by Cheng Shao at 2025-12-20T17:06:51-05:00 hadrian: add with_profiled_libs flavour transformer This patch adds a `with_profiled_libs` flavour transformer to hadrian which is the exact opposite of `no_profiled_libs`. It adds profiling ways to stage1+ rts/library ways, and doesn't alter other flavour settings. It is useful when needing to test profiling logic locally with a quick flavour. - - - - - 746b18cd by Cheng Shao at 2025-12-20T17:06:51-05:00 hadrian: fix missing profiled dynamic libraries in profiled_ghc This commit fixes the profiled_ghc flavour transformer to include profiled dynamic libraries as well, since they're supported by GHC since !12595. - - - - - 4dd7e3b9 by Cheng Shao at 2025-12-20T17:07:33-05:00 ci: set http.postBuffer to mitigate perf notes timeout on some runners This patch sets http.postBuffer to mitigate the timeout when fetching perf notes on some runners with slow internet connection. Fixes #26684. - - - - - bc36268a by Wolfgang Jeltsch at 2025-12-21T16:23:24-05:00 Remove unused known keys and names for type representations This removes the known-key and corresponding name variables for `TrName`, `TrNameD`, `TypeRep`, `KindRepTypeLitD`, `TypeLitSort`, and `mkTrType`, as they are apparently nowhere used in GHC’s source code. - - - - - ff5050e9 by Wolfgang Jeltsch at 2025-12-21T16:24:04-05:00 Remove unused known keys and names for natural operations This removes the known-key and corresponding name variables for `naturalAndNot`, `naturalLog2`, `naturalLogBaseWord`, `naturalLogBase`, `naturalPowMod`, `naturalSizeInBase`, `naturalToFloat`, and `naturalToDouble`, as they are apparently nowhere used in GHC’s source code. - - - - - 424388c2 by Wolfgang Jeltsch at 2025-12-21T16:24:45-05:00 Remove the unused known key and name for `Fingerprint` This removes the variables for the known key and the name of the `Fingerprint` data constructor, as they are apparently nowhere used in GHC’s source code. - - - - - a1ed86fe by Wolfgang Jeltsch at 2025-12-21T16:25:26-05:00 Remove the unused known key and name for `failIO` This removes the variables for the known key and the name of the `failIO` operation, as they are apparently nowhere used in GHC’s source code. - - - - - b8220daf by Wolfgang Jeltsch at 2025-12-21T16:26:07-05:00 Remove the unused known key and name for `liftM` This removes the variables for the known key and the name of the `liftM` operation, as they are apparently nowhere used in GHC’s source code. - - - - - eb0628b1 by Wolfgang Jeltsch at 2025-12-21T16:26:47-05:00 Fix the documentation of `hIsClosed` - - - - - 0b8db563 by Cheng Shao at 2025-12-22T06:48:54+01:00 rts: add is-valid-utf8.c to .ubsan-suppressions A minor one in `bytestring` that might surface when building with +ubsan using clang. - - - - - 0e61bfec by Cheng Shao at 2025-12-22T06:48:54+01:00 hadrian: add support for building with AddressSanitizer This patch adds a +asan flavour transformer to hadrian to build all stage1+ C/C++ code with AddressBehaviorSanitizer. This is particularly useful to catch potential out-of-bounds and use-after-free bugs in the RTS codebase. - - - - - 44aff5d9 by Cheng Shao at 2025-12-22T06:48:54+01:00 ci: add ubsan+asan job We now have a `x86_64-linux-fedora43-validate+debug_info+ubsan+asan` validate/nightly job with both UBSan/ASan enabled. - - - - - 71b381b4 by Cheng Shao at 2025-12-22T06:48:55+01:00 rts: add ASAN instrumentation to mblock allocator - - - - - ebcb7265 by Cheng Shao at 2025-12-22T06:48:55+01:00 rts: add ASAN instrumentation to per-Task InCall free list - - - - - 30 changed files: - .gitlab/ci.sh - .gitlab/generate-ci/gen_ci.hs - .gitlab/jobs.yaml - compiler/GHC/Builtin/Names.hs - compiler/GHC/HsToCore/Foreign/Wasm.hs - compiler/GHC/Parser/String.hs - compiler/GHC/StgToCmm/InfoTableProv.hs - docs/users_guide/wasm.rst - hadrian/doc/flavours.md - hadrian/src/Flavour.hs - hadrian/src/Settings/Packages.hs - libraries/ghc-internal/src/GHC/Internal/IO/Handle.hs - rts/.ubsan-suppressions - rts/Task.c - rts/include/Stg.h - + rts/include/rts/ASANUtils.h - rts/linker/InitFini.c - rts/rts.cabal - rts/sm/MBlock.c - rts/sm/Sanity.c - testsuite/driver/testglobals.py - testsuite/driver/testlib.py - testsuite/tests/ffi/should_run/all.T - testsuite/tests/jsffi/all.T - + testsuite/tests/jsffi/bytearrayarg.hs - + testsuite/tests/jsffi/bytearrayarg.mjs - + testsuite/tests/jsffi/bytearrayarg.stdout - testsuite/tests/perf/should_run/all.T - testsuite/tests/rts/T18623/all.T - testsuite/tests/rts/all.T Changes: ===================================== .gitlab/ci.sh ===================================== @@ -265,6 +265,15 @@ function setup() { # testsuite driver! git config gc.auto 0 + # Some runners still choke at the perf note fetch step, which has to + # do with slow internet connection, see + # https://docs.gitlab.com/topics/git/troubleshooting_git/#error-stream-0-was-n... + # for the http.postBuffer mitigation. It might seem + # counter-intuitive that "post buffer" helps with fetching, but git + # indeed issues post requests when fetching over https, it's a + # bidirectional negotiation with the remote. + git config http.postBuffer 52428800 + info "=====================================================" info "Toolchain versions" info "=====================================================" ===================================== .gitlab/generate-ci/gen_ci.hs ===================================== @@ -162,6 +162,7 @@ data BuildConfig , tablesNextToCode :: Bool , threadSanitiser :: Bool , ubsan :: Bool + , asan :: Bool , noSplitSections :: Bool , validateNonmovingGc :: Bool , textWithSIMDUTF :: Bool @@ -173,7 +174,7 @@ configureArgsStr :: BuildConfig -> String configureArgsStr bc = unwords $ ["--enable-unregisterised"| unregisterised bc ] ++ ["--disable-tables-next-to-code" | not (tablesNextToCode bc) ] - ++ ["--with-intree-gmp" | Just _ <- pure (crossTarget bc) ] + ++ ["--with-intree-gmp" | isJust (crossTarget bc) || ubsan bc || asan bc ] ++ ["--with-system-libffi" | crossTarget bc == Just "wasm32-wasi" ] ++ ["--enable-ipe-data-compression" | withZstd bc ] ++ ["--enable-strict-ghc-toolchain-check"] @@ -188,6 +189,7 @@ mkJobFlavour BuildConfig{..} = Flavour buildFlavour opts [HostFullyStatic | hostFullyStatic] ++ [ThreadSanitiser | threadSanitiser] ++ [UBSan | ubsan] ++ + [ASan | asan] ++ [NoSplitSections | noSplitSections, buildFlavour == Release ] ++ [BootNonmovingGc | validateNonmovingGc ] ++ [TextWithSIMDUTF | textWithSIMDUTF] @@ -201,6 +203,7 @@ data FlavourTrans = | HostFullyStatic | ThreadSanitiser | UBSan + | ASan | NoSplitSections | BootNonmovingGc | TextWithSIMDUTF @@ -230,6 +233,7 @@ vanilla = BuildConfig , tablesNextToCode = True , threadSanitiser = False , ubsan = False + , asan = False , noSplitSections = False , validateNonmovingGc = False , textWithSIMDUTF = False @@ -283,8 +287,8 @@ llvm = vanilla { llvmBootstrap = True } tsan :: BuildConfig tsan = vanilla { threadSanitiser = True } -enableUBSan :: BuildConfig -enableUBSan = vanilla { withDwarf = True, ubsan = True } +enableUBSanASan :: BuildConfig +enableUBSanASan = vanilla { withDwarf = True, ubsan = True, asan = True } noTntc :: BuildConfig noTntc = vanilla { tablesNextToCode = False } @@ -381,6 +385,7 @@ flavourString (Flavour base trans) = base_string base ++ concatMap (("+" ++) . f flavour_string HostFullyStatic = "host_fully_static" flavour_string ThreadSanitiser = "thread_sanitizer_cmm" flavour_string UBSan = "ubsan" + flavour_string ASan = "asan" flavour_string NoSplitSections = "no_split_sections" flavour_string BootNonmovingGc = "boot_nonmoving_gc" flavour_string TextWithSIMDUTF = "text_simdutf" @@ -1213,15 +1218,24 @@ fedora_x86 = , hackage_doc_job (disableValidate (standardBuildsWithConfig Amd64 (Linux Fedora43) releaseConfig)) , disableValidate (standardBuildsWithConfig Amd64 (Linux Fedora43) dwarf) , disableValidate (standardBuilds Amd64 (Linux Fedora43)) - -- For UBSan jobs, only enable for validate/nightly pipelines. - -- Also disable docs since it's not the point for UBSan jobs. + -- For UBSan/ASan jobs, only enable for validate/nightly + -- pipelines. Also disable docs since it's not the point for + -- UBSan/ASan jobs. + -- + -- See + -- https://github.com/llvm/llvm-project/blob/llvmorg-21.1.8/compiler-rt/lib/san... + -- for ASAN options help, for now these are required to pass the + -- testsuite , modifyJobs ( setVariable "HADRIAN_ARGS" "--docs=none" . addVariable "UBSAN_OPTIONS" "suppressions=$CI_PROJECT_DIR/rts/.ubsan-suppressions" + . addVariable + "ASAN_OPTIONS" + "detect_leaks=false:handle_segv=0:handle_sigfpe=0:verify_asan_link_order=false" ) - $ validateBuilds Amd64 (Linux Fedora43) enableUBSan + $ validateBuilds Amd64 (Linux Fedora43) enableUBSanASan ] where hackage_doc_job = rename (<> "-hackage") . modifyJobs (addVariable "HADRIAN_ARGS" "--haddock-for-hackage") ===================================== .gitlab/jobs.yaml ===================================== @@ -3195,7 +3195,7 @@ "XZ_OPT": "-9" } }, - "nightly-x86_64-linux-fedora43-validate+debug_info+ubsan": { + "nightly-x86_64-linux-fedora43-validate+debug_info+ubsan+asan": { "after_script": [ ".gitlab/ci.sh save_cache", ".gitlab/ci.sh save_test_output", @@ -3206,7 +3206,7 @@ "artifacts": { "expire_in": "8 weeks", "paths": [ - "ghc-x86_64-linux-fedora43-validate+debug_info+ubsan.tar.xz", + "ghc-x86_64-linux-fedora43-validate+debug_info+ubsan+asan.tar.xz", "junit.xml", "unexpected-test-output.tar.gz" ], @@ -3248,14 +3248,15 @@ "x86_64-linux" ], "variables": { + "ASAN_OPTIONS": "detect_leaks=false:handle_segv=0:handle_sigfpe=0:verify_asan_link_order=false", "BIGNUM_BACKEND": "gmp", - "BIN_DIST_NAME": "ghc-x86_64-linux-fedora43-validate+debug_info+ubsan", - "BUILD_FLAVOUR": "validate+debug_info+ubsan", - "CONFIGURE_ARGS": "--enable-strict-ghc-toolchain-check", + "BIN_DIST_NAME": "ghc-x86_64-linux-fedora43-validate+debug_info+ubsan+asan", + "BUILD_FLAVOUR": "validate+debug_info+ubsan+asan", + "CONFIGURE_ARGS": "--with-intree-gmp --enable-strict-ghc-toolchain-check", "HADRIAN_ARGS": "--docs=none", "INSTALL_CONFIGURE_ARGS": "--enable-strict-ghc-toolchain-check", "RUNTEST_ARGS": "", - "TEST_ENV": "x86_64-linux-fedora43-validate+debug_info+ubsan", + "TEST_ENV": "x86_64-linux-fedora43-validate+debug_info+ubsan+asan", "UBSAN_OPTIONS": "suppressions=$CI_PROJECT_DIR/rts/.ubsan-suppressions", "XZ_OPT": "-9" } @@ -7346,7 +7347,7 @@ "TEST_ENV": "x86_64-linux-fedora43-validate+debug_info" } }, - "x86_64-linux-fedora43-validate+debug_info+ubsan": { + "x86_64-linux-fedora43-validate+debug_info+ubsan+asan": { "after_script": [ ".gitlab/ci.sh save_cache", ".gitlab/ci.sh save_test_output", @@ -7357,7 +7358,7 @@ "artifacts": { "expire_in": "2 weeks", "paths": [ - "ghc-x86_64-linux-fedora43-validate+debug_info+ubsan.tar.xz", + "ghc-x86_64-linux-fedora43-validate+debug_info+ubsan+asan.tar.xz", "junit.xml", "unexpected-test-output.tar.gz" ], @@ -7383,7 +7384,7 @@ ], "rules": [ { - "if": "((($ONLY_JOBS) && ($ONLY_JOBS =~ /.*\\bx86_64-linux-fedora43-validate\\+debug_info\\+ubsan(\\s|$).*/)) || (($ONLY_JOBS == null) && ((($CI_MERGE_REQUEST_LABELS =~ /.*full-ci.*/) || ($CI_MERGE_REQUEST_LABELS =~ /.*marge_bot_batch_merge_job.*/) || ($CI_COMMIT_BRANCH == \"master\") || ($CI_COMMIT_BRANCH =~ /ghc-[0-9]+\\.[0-9]+/))))) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null)", + "if": "((($ONLY_JOBS) && ($ONLY_JOBS =~ /.*\\bx86_64-linux-fedora43-validate\\+debug_info\\+ubsan\\+asan(\\s|$).*/)) || (($ONLY_JOBS == null) && ((($CI_MERGE_REQUEST_LABELS =~ /.*full-ci.*/) || ($CI_MERGE_REQUEST_LABELS =~ /.*marge_bot_batch_merge_job.*/) || ($CI_COMMIT_BRANCH == \"master\") || ($CI_COMMIT_BRANCH =~ /ghc-[0-9]+\\.[0-9]+/))))) && ($RELEASE_JOB != \"yes\") && ($NIGHTLY == null)", "when": "on_success" } ], @@ -7399,14 +7400,15 @@ "x86_64-linux" ], "variables": { + "ASAN_OPTIONS": "detect_leaks=false:handle_segv=0:handle_sigfpe=0:verify_asan_link_order=false", "BIGNUM_BACKEND": "gmp", - "BIN_DIST_NAME": "ghc-x86_64-linux-fedora43-validate+debug_info+ubsan", - "BUILD_FLAVOUR": "validate+debug_info+ubsan", - "CONFIGURE_ARGS": "--enable-strict-ghc-toolchain-check", + "BIN_DIST_NAME": "ghc-x86_64-linux-fedora43-validate+debug_info+ubsan+asan", + "BUILD_FLAVOUR": "validate+debug_info+ubsan+asan", + "CONFIGURE_ARGS": "--with-intree-gmp --enable-strict-ghc-toolchain-check", "HADRIAN_ARGS": "--docs=none", "INSTALL_CONFIGURE_ARGS": "--enable-strict-ghc-toolchain-check", "RUNTEST_ARGS": "", - "TEST_ENV": "x86_64-linux-fedora43-validate+debug_info+ubsan", + "TEST_ENV": "x86_64-linux-fedora43-validate+debug_info+ubsan+asan", "UBSAN_OPTIONS": "suppressions=$CI_PROJECT_DIR/rts/.ubsan-suppressions" } }, ===================================== compiler/GHC/Builtin/Names.hs ===================================== @@ -222,12 +222,11 @@ basicKnownKeyNames -- Type representation types trModuleTyConName, trModuleDataConName, - trNameTyConName, trNameSDataConName, trNameDDataConName, + trNameSDataConName, trTyConTyConName, trTyConDataConName, -- Typeable typeableClassName, - typeRepTyConName, someTypeRepTyConName, someTypeRepDataConName, kindRepTyConName, @@ -237,13 +236,10 @@ basicKnownKeyNames kindRepFunDataConName, kindRepTYPEDataConName, kindRepTypeLitSDataConName, - kindRepTypeLitDDataConName, - typeLitSortTyConName, typeLitSymbolDataConName, typeLitNatDataConName, typeLitCharDataConName, typeRepIdName, - mkTrTypeName, mkTrConName, mkTrAppCheckedName, mkTrFunName, @@ -296,7 +292,7 @@ basicKnownKeyNames fmapName, -- Monad stuff - thenIOName, bindIOName, returnIOName, failIOName, bindMName, thenMName, + thenIOName, bindIOName, returnIOName, bindMName, thenMName, returnMName, joinMName, -- MonadFail @@ -409,26 +405,18 @@ basicKnownKeyNames naturalQuotName, naturalRemName, naturalAndName, - naturalAndNotName, naturalOrName, naturalXorName, naturalTestBitName, naturalBitName, naturalGcdName, naturalLcmName, - naturalLog2Name, - naturalLogBaseWordName, - naturalLogBaseName, - naturalPowModName, - naturalSizeInBaseName, bignatEqName, -- Float/Double integerToFloatName, integerToDoubleName, - naturalToFloatName, - naturalToDoubleName, rationalToFloatName, rationalToDoubleName, @@ -476,11 +464,9 @@ basicKnownKeyNames -- Generics , genClassName, gen1ClassName - , datatypeClassName, constructorClassName, selectorClassName -- Monad comprehensions , guardMName - , liftMName , mzipName -- GHCi Sandbox @@ -492,9 +478,6 @@ basicKnownKeyNames , staticPtrDataConName, staticPtrInfoDataConName , fromStaticPtrName - -- Fingerprint - , fingerprintDataConName - -- Custom type errors , errorMessageTypeErrorFamName , typeErrorTextDataConName @@ -517,12 +500,9 @@ basicKnownKeyNames genericTyConNames :: [Name] genericTyConNames = [ - v1TyConName, u1TyConName, par1TyConName, rec1TyConName, - k1TyConName, m1TyConName, sumTyConName, prodTyConName, - compTyConName, rTyConName, dTyConName, - cTyConName, sTyConName, rec0TyConName, - d1TyConName, c1TyConName, s1TyConName, - repTyConName, rep1TyConName, uRecTyConName, + v1TyConName, u1TyConName, par1TyConName, rec1TyConName, sumTyConName, + prodTyConName, compTyConName, rec0TyConName, d1TyConName, c1TyConName, + s1TyConName, repTyConName, rep1TyConName, uAddrTyConName, uCharTyConName, uDoubleTyConName, uFloatTyConName, uIntTyConName, uWordTyConName, prefixIDataConName, infixIDataConName, leftAssociativeDataConName, @@ -939,11 +919,8 @@ voidTyConName = tcQual gHC_INTERNAL_BASE (fsLit "Void") voidTyConKey -- Generics (types) v1TyConName, u1TyConName, par1TyConName, rec1TyConName, - k1TyConName, m1TyConName, sumTyConName, prodTyConName, - compTyConName, rTyConName, dTyConName, - cTyConName, sTyConName, rec0TyConName, - d1TyConName, c1TyConName, s1TyConName, - repTyConName, rep1TyConName, uRecTyConName, + sumTyConName, prodTyConName, compTyConName, rec0TyConName, d1TyConName, + c1TyConName, s1TyConName, repTyConName, rep1TyConName, uAddrTyConName, uCharTyConName, uDoubleTyConName, uFloatTyConName, uIntTyConName, uWordTyConName, prefixIDataConName, infixIDataConName, leftAssociativeDataConName, @@ -958,18 +935,11 @@ v1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "V1") v1TyConKey u1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "U1") u1TyConKey par1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "Par1") par1TyConKey rec1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "Rec1") rec1TyConKey -k1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "K1") k1TyConKey -m1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "M1") m1TyConKey sumTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit ":+:") sumTyConKey prodTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit ":*:") prodTyConKey compTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit ":.:") compTyConKey -rTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "R") rTyConKey -dTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "D") dTyConKey -cTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "C") cTyConKey -sTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "S") sTyConKey - rec0TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "Rec0") rec0TyConKey d1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "D1") d1TyConKey c1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "C1") c1TyConKey @@ -978,7 +948,6 @@ s1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "S1") s1TyConKey repTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "Rep") repTyConKey rep1TyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "Rep1") rep1TyConKey -uRecTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "URec") uRecTyConKey uAddrTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "UAddr") uAddrTyConKey uCharTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "UChar") uCharTyConKey uDoubleTyConName = tcQual gHC_INTERNAL_GENERICS (fsLit "UDouble") uDoubleTyConKey @@ -1169,18 +1138,12 @@ integerFromNaturalName , naturalQuotName , naturalRemName , naturalAndName - , naturalAndNotName , naturalOrName , naturalXorName , naturalTestBitName , naturalBitName , naturalGcdName , naturalLcmName - , naturalLog2Name - , naturalLogBaseWordName - , naturalLogBaseName - , naturalPowModName - , naturalSizeInBaseName , bignatEqName , bignatCompareName , bignatCompareWordName @@ -1209,18 +1172,12 @@ naturalQuotRemName = bnnVarQual "naturalQuotRem#" naturalQuotRe naturalQuotName = bnnVarQual "naturalQuot" naturalQuotIdKey naturalRemName = bnnVarQual "naturalRem" naturalRemIdKey naturalAndName = bnnVarQual "naturalAnd" naturalAndIdKey -naturalAndNotName = bnnVarQual "naturalAndNot" naturalAndNotIdKey naturalOrName = bnnVarQual "naturalOr" naturalOrIdKey naturalXorName = bnnVarQual "naturalXor" naturalXorIdKey naturalTestBitName = bnnVarQual "naturalTestBit#" naturalTestBitIdKey naturalBitName = bnnVarQual "naturalBit#" naturalBitIdKey naturalGcdName = bnnVarQual "naturalGcd" naturalGcdIdKey naturalLcmName = bnnVarQual "naturalLcm" naturalLcmIdKey -naturalLog2Name = bnnVarQual "naturalLog2#" naturalLog2IdKey -naturalLogBaseWordName = bnnVarQual "naturalLogBaseWord#" naturalLogBaseWordIdKey -naturalLogBaseName = bnnVarQual "naturalLogBase#" naturalLogBaseIdKey -naturalPowModName = bnnVarQual "naturalPowMod" naturalPowModIdKey -naturalSizeInBaseName = bnnVarQual "naturalSizeInBase#" naturalSizeInBaseIdKey integerFromNaturalName = bniVarQual "integerFromNatural" integerFromNaturalIdKey integerToNaturalClampName = bniVarQual "integerToNaturalClamp" integerToNaturalClampIdKey @@ -1291,12 +1248,9 @@ realFloatClassName = clsQual gHC_INTERNAL_FLOAT (fsLit "RealFloat") realFloatCla -- other GHC.Internal.Float functions integerToFloatName, integerToDoubleName, - naturalToFloatName, naturalToDoubleName, rationalToFloatName, rationalToDoubleName :: Name integerToFloatName = varQual gHC_INTERNAL_FLOAT (fsLit "integerToFloat#") integerToFloatIdKey integerToDoubleName = varQual gHC_INTERNAL_FLOAT (fsLit "integerToDouble#") integerToDoubleIdKey -naturalToFloatName = varQual gHC_INTERNAL_FLOAT (fsLit "naturalToFloat#") naturalToFloatIdKey -naturalToDoubleName = varQual gHC_INTERNAL_FLOAT (fsLit "naturalToDouble#") naturalToDoubleIdKey rationalToFloatName = varQual gHC_INTERNAL_FLOAT (fsLit "rationalToFloat") rationalToFloatIdKey rationalToDoubleName = varQual gHC_INTERNAL_FLOAT (fsLit "rationalToDouble") rationalToDoubleIdKey @@ -1307,17 +1261,13 @@ ixClassName = clsQual gHC_INTERNAL_IX (fsLit "Ix") ixClassKey -- Typeable representation types trModuleTyConName , trModuleDataConName - , trNameTyConName , trNameSDataConName - , trNameDDataConName , trTyConTyConName , trTyConDataConName :: Name trModuleTyConName = tcQual gHC_TYPES (fsLit "Module") trModuleTyConKey trModuleDataConName = dcQual gHC_TYPES (fsLit "Module") trModuleDataConKey -trNameTyConName = tcQual gHC_TYPES (fsLit "TrName") trNameTyConKey trNameSDataConName = dcQual gHC_TYPES (fsLit "TrNameS") trNameSDataConKey -trNameDDataConName = dcQual gHC_TYPES (fsLit "TrNameD") trNameDDataConKey trTyConTyConName = tcQual gHC_TYPES (fsLit "TyCon") trTyConTyConKey trTyConDataConName = dcQual gHC_TYPES (fsLit "TyCon") trTyConDataConKey @@ -1328,7 +1278,6 @@ kindRepTyConName , kindRepFunDataConName , kindRepTYPEDataConName , kindRepTypeLitSDataConName - , kindRepTypeLitDDataConName :: Name kindRepTyConName = tcQual gHC_TYPES (fsLit "KindRep") kindRepTyConKey kindRepTyConAppDataConName = dcQual gHC_TYPES (fsLit "KindRepTyConApp") kindRepTyConAppDataConKey @@ -1337,24 +1286,19 @@ kindRepAppDataConName = dcQual gHC_TYPES (fsLit "KindRepApp") kindR kindRepFunDataConName = dcQual gHC_TYPES (fsLit "KindRepFun") kindRepFunDataConKey kindRepTYPEDataConName = dcQual gHC_TYPES (fsLit "KindRepTYPE") kindRepTYPEDataConKey kindRepTypeLitSDataConName = dcQual gHC_TYPES (fsLit "KindRepTypeLitS") kindRepTypeLitSDataConKey -kindRepTypeLitDDataConName = dcQual gHC_TYPES (fsLit "KindRepTypeLitD") kindRepTypeLitDDataConKey -typeLitSortTyConName - , typeLitSymbolDataConName +typeLitSymbolDataConName , typeLitNatDataConName , typeLitCharDataConName :: Name -typeLitSortTyConName = tcQual gHC_TYPES (fsLit "TypeLitSort") typeLitSortTyConKey typeLitSymbolDataConName = dcQual gHC_TYPES (fsLit "TypeLitSymbol") typeLitSymbolDataConKey typeLitNatDataConName = dcQual gHC_TYPES (fsLit "TypeLitNat") typeLitNatDataConKey typeLitCharDataConName = dcQual gHC_TYPES (fsLit "TypeLitChar") typeLitCharDataConKey -- Class Typeable, and functions for constructing `Typeable` dictionaries typeableClassName - , typeRepTyConName , someTypeRepTyConName , someTypeRepDataConName - , mkTrTypeName , mkTrConName , mkTrAppCheckedName , mkTrFunName @@ -1365,11 +1309,9 @@ typeableClassName , trGhcPrimModuleName :: Name typeableClassName = clsQual gHC_INTERNAL_TYPEABLE_INTERNAL (fsLit "Typeable") typeableClassKey -typeRepTyConName = tcQual gHC_INTERNAL_TYPEABLE_INTERNAL (fsLit "TypeRep") typeRepTyConKey someTypeRepTyConName = tcQual gHC_INTERNAL_TYPEABLE_INTERNAL (fsLit "SomeTypeRep") someTypeRepTyConKey someTypeRepDataConName = dcQual gHC_INTERNAL_TYPEABLE_INTERNAL (fsLit "SomeTypeRep") someTypeRepDataConKey typeRepIdName = varQual gHC_INTERNAL_TYPEABLE_INTERNAL (fsLit "typeRep#") typeRepIdKey -mkTrTypeName = varQual gHC_INTERNAL_TYPEABLE_INTERNAL (fsLit "mkTrType") mkTrTypeKey mkTrConName = varQual gHC_INTERNAL_TYPEABLE_INTERNAL (fsLit "mkTrCon") mkTrConKey mkTrAppCheckedName = varQual gHC_INTERNAL_TYPEABLE_INTERNAL (fsLit "mkTrAppChecked") mkTrAppCheckedKey mkTrFunName = varQual gHC_INTERNAL_TYPEABLE_INTERNAL (fsLit "mkTrFun") mkTrFunKey @@ -1494,15 +1436,10 @@ readClassName :: Name readClassName = clsQual gHC_INTERNAL_READ (fsLit "Read") readClassKey -- Classes Generic and Generic1, Datatype, Constructor and Selector -genClassName, gen1ClassName, datatypeClassName, constructorClassName, - selectorClassName :: Name +genClassName, gen1ClassName :: Name genClassName = clsQual gHC_INTERNAL_GENERICS (fsLit "Generic") genClassKey gen1ClassName = clsQual gHC_INTERNAL_GENERICS (fsLit "Generic1") gen1ClassKey -datatypeClassName = clsQual gHC_INTERNAL_GENERICS (fsLit "Datatype") datatypeClassKey -constructorClassName = clsQual gHC_INTERNAL_GENERICS (fsLit "Constructor") constructorClassKey -selectorClassName = clsQual gHC_INTERNAL_GENERICS (fsLit "Selector") selectorClassKey - genericClassNames :: [Name] genericClassNames = [genClassName, gen1ClassName] @@ -1513,13 +1450,12 @@ ghciStepIoMName = varQual gHC_INTERNAL_GHCI (fsLit "ghciStepIO") ghciStepIoMClas -- IO things ioTyConName, ioDataConName, - thenIOName, bindIOName, returnIOName, failIOName :: Name + thenIOName, bindIOName, returnIOName :: Name ioTyConName = tcQual gHC_TYPES (fsLit "IO") ioTyConKey ioDataConName = dcQual gHC_TYPES (fsLit "IO") ioDataConKey thenIOName = varQual gHC_INTERNAL_BASE (fsLit "thenIO") thenIOIdKey bindIOName = varQual gHC_INTERNAL_BASE (fsLit "bindIO") bindIOIdKey returnIOName = varQual gHC_INTERNAL_BASE (fsLit "returnIO") returnIOIdKey -failIOName = varQual gHC_INTERNAL_IO (fsLit "failIO") failIOIdKey -- IO things printName :: Name @@ -1564,9 +1500,8 @@ choiceAName = varQual gHC_INTERNAL_ARROW (fsLit "|||") choiceAIdKey loopAName = varQual gHC_INTERNAL_ARROW (fsLit "loop") loopAIdKey -- Monad comprehensions -guardMName, liftMName, mzipName :: Name +guardMName, mzipName :: Name guardMName = varQual gHC_INTERNAL_MONAD (fsLit "guard") guardMIdKey -liftMName = varQual gHC_INTERNAL_MONAD (fsLit "liftM") liftMIdKey mzipName = varQual gHC_INTERNAL_CONTROL_MONAD_ZIP (fsLit "mzip") mzipIdKey @@ -1654,10 +1589,6 @@ fromStaticPtrName :: Name fromStaticPtrName = varQual gHC_INTERNAL_STATICPTR (fsLit "fromStaticPtr") fromStaticPtrClassOpKey -fingerprintDataConName :: Name -fingerprintDataConName = - dcQual gHC_INTERNAL_FINGERPRINT_TYPE (fsLit "Fingerprint") fingerprintDataConKey - constPtrConName :: Name constPtrConName = tcQual gHC_INTERNAL_FOREIGN_C_CONSTPTR (fsLit "ConstPtr") constPtrTyConKey @@ -1753,15 +1684,10 @@ applicativeClassKey = mkPreludeClassUnique 34 foldableClassKey = mkPreludeClassUnique 35 traversableClassKey = mkPreludeClassUnique 36 -genClassKey, gen1ClassKey, datatypeClassKey, constructorClassKey, - selectorClassKey :: Unique +genClassKey, gen1ClassKey :: Unique genClassKey = mkPreludeClassUnique 37 gen1ClassKey = mkPreludeClassUnique 38 -datatypeClassKey = mkPreludeClassUnique 39 -constructorClassKey = mkPreludeClassUnique 40 -selectorClassKey = mkPreludeClassUnique 41 - -- KnownNat: see Note [KnownNat & KnownSymbol and EvLit] in GHC.Tc.Instance.Class knownNatClassNameKey :: Unique knownNatClassNameKey = mkPreludeClassUnique 42 @@ -1940,21 +1866,16 @@ pluginTyConKey, frontendPluginTyConKey :: Unique pluginTyConKey = mkPreludeTyConUnique 102 frontendPluginTyConKey = mkPreludeTyConUnique 103 -trTyConTyConKey, trModuleTyConKey, trNameTyConKey, - kindRepTyConKey, typeLitSortTyConKey :: Unique +trTyConTyConKey, trModuleTyConKey, + kindRepTyConKey :: Unique trTyConTyConKey = mkPreludeTyConUnique 104 trModuleTyConKey = mkPreludeTyConUnique 105 -trNameTyConKey = mkPreludeTyConUnique 106 kindRepTyConKey = mkPreludeTyConUnique 107 -typeLitSortTyConKey = mkPreludeTyConUnique 108 -- Generics (Unique keys) v1TyConKey, u1TyConKey, par1TyConKey, rec1TyConKey, - k1TyConKey, m1TyConKey, sumTyConKey, prodTyConKey, - compTyConKey, rTyConKey, dTyConKey, - cTyConKey, sTyConKey, rec0TyConKey, - d1TyConKey, c1TyConKey, s1TyConKey, - repTyConKey, rep1TyConKey, uRecTyConKey, + sumTyConKey, prodTyConKey, compTyConKey, rec0TyConKey, + d1TyConKey, c1TyConKey, s1TyConKey, repTyConKey, rep1TyConKey, uAddrTyConKey, uCharTyConKey, uDoubleTyConKey, uFloatTyConKey, uIntTyConKey, uWordTyConKey :: Unique @@ -1962,18 +1883,11 @@ v1TyConKey = mkPreludeTyConUnique 135 u1TyConKey = mkPreludeTyConUnique 136 par1TyConKey = mkPreludeTyConUnique 137 rec1TyConKey = mkPreludeTyConUnique 138 -k1TyConKey = mkPreludeTyConUnique 139 -m1TyConKey = mkPreludeTyConUnique 140 sumTyConKey = mkPreludeTyConUnique 141 prodTyConKey = mkPreludeTyConUnique 142 compTyConKey = mkPreludeTyConUnique 143 -rTyConKey = mkPreludeTyConUnique 144 -dTyConKey = mkPreludeTyConUnique 146 -cTyConKey = mkPreludeTyConUnique 147 -sTyConKey = mkPreludeTyConUnique 148 - rec0TyConKey = mkPreludeTyConUnique 149 d1TyConKey = mkPreludeTyConUnique 151 c1TyConKey = mkPreludeTyConUnique 152 @@ -1982,7 +1896,6 @@ s1TyConKey = mkPreludeTyConUnique 153 repTyConKey = mkPreludeTyConUnique 155 rep1TyConKey = mkPreludeTyConUnique 156 -uRecTyConKey = mkPreludeTyConUnique 157 uAddrTyConKey = mkPreludeTyConUnique 158 uCharTyConKey = mkPreludeTyConUnique 159 uDoubleTyConKey = mkPreludeTyConUnique 160 @@ -2026,8 +1939,7 @@ callStackTyConKey :: Unique callStackTyConKey = mkPreludeTyConUnique 191 -- Typeables -typeRepTyConKey, someTypeRepTyConKey, someTypeRepDataConKey :: Unique -typeRepTyConKey = mkPreludeTyConUnique 192 +someTypeRepTyConKey, someTypeRepDataConKey :: Unique someTypeRepTyConKey = mkPreludeTyConUnique 193 someTypeRepDataConKey = mkPreludeTyConUnique 194 @@ -2159,19 +2071,15 @@ staticPtrDataConKey = mkPreludeDataConUnique 33 staticPtrInfoDataConKey :: Unique staticPtrInfoDataConKey = mkPreludeDataConUnique 34 -fingerprintDataConKey :: Unique -fingerprintDataConKey = mkPreludeDataConUnique 35 - srcLocDataConKey :: Unique srcLocDataConKey = mkPreludeDataConUnique 37 trTyConDataConKey, trModuleDataConKey, - trNameSDataConKey, trNameDDataConKey, + trNameSDataConKey, trGhcPrimModuleKey :: Unique trTyConDataConKey = mkPreludeDataConUnique 41 trModuleDataConKey = mkPreludeDataConUnique 43 trNameSDataConKey = mkPreludeDataConUnique 45 -trNameDDataConKey = mkPreludeDataConUnique 46 trGhcPrimModuleKey = mkPreludeDataConUnique 47 typeErrorTextDataConKey, @@ -2246,7 +2154,7 @@ vecElemDataConKeys = map mkPreludeDataConUnique [96..105] -- Typeable things kindRepTyConAppDataConKey, kindRepVarDataConKey, kindRepAppDataConKey, kindRepFunDataConKey, kindRepTYPEDataConKey, - kindRepTypeLitSDataConKey, kindRepTypeLitDDataConKey + kindRepTypeLitSDataConKey :: Unique kindRepTyConAppDataConKey = mkPreludeDataConUnique 106 kindRepVarDataConKey = mkPreludeDataConUnique 107 @@ -2254,7 +2162,6 @@ kindRepAppDataConKey = mkPreludeDataConUnique 108 kindRepFunDataConKey = mkPreludeDataConUnique 109 kindRepTYPEDataConKey = mkPreludeDataConUnique 110 kindRepTypeLitSDataConKey = mkPreludeDataConUnique 111 -kindRepTypeLitDDataConKey = mkPreludeDataConUnique 112 typeLitSymbolDataConKey, typeLitNatDataConKey, typeLitCharDataConKey :: Unique typeLitSymbolDataConKey = mkPreludeDataConUnique 113 @@ -2342,7 +2249,7 @@ cstringLengthIdKey = mkPreludeMiscIdUnique 28 concatIdKey, filterIdKey, zipIdKey, bindIOIdKey, returnIOIdKey, newStablePtrIdKey, - printIdKey, failIOIdKey, nullAddrIdKey, voidArgIdKey, + printIdKey, nullAddrIdKey, voidArgIdKey, otherwiseIdKey, assertIdKey :: Unique concatIdKey = mkPreludeMiscIdUnique 31 filterIdKey = mkPreludeMiscIdUnique 32 @@ -2351,7 +2258,6 @@ bindIOIdKey = mkPreludeMiscIdUnique 34 returnIOIdKey = mkPreludeMiscIdUnique 35 newStablePtrIdKey = mkPreludeMiscIdUnique 36 printIdKey = mkPreludeMiscIdUnique 37 -failIOIdKey = mkPreludeMiscIdUnique 38 nullAddrIdKey = mkPreludeMiscIdUnique 39 voidArgIdKey = mkPreludeMiscIdUnique 40 otherwiseIdKey = mkPreludeMiscIdUnique 43 @@ -2390,11 +2296,9 @@ considerAccessibleIdKey = mkPreludeMiscIdUnique 125 noinlineIdKey = mkPreludeMiscIdUnique 126 noinlineConstraintIdKey = mkPreludeMiscIdUnique 127 -integerToFloatIdKey, integerToDoubleIdKey, naturalToFloatIdKey, naturalToDoubleIdKey :: Unique +integerToFloatIdKey, integerToDoubleIdKey :: Unique integerToFloatIdKey = mkPreludeMiscIdUnique 128 integerToDoubleIdKey = mkPreludeMiscIdUnique 129 -naturalToFloatIdKey = mkPreludeMiscIdUnique 130 -naturalToDoubleIdKey = mkPreludeMiscIdUnique 131 rationalToFloatIdKey, rationalToDoubleIdKey :: Unique rationalToFloatIdKey = mkPreludeMiscIdUnique 132 @@ -2472,9 +2376,8 @@ toIntegerClassOpKey = mkPreludeMiscIdUnique 192 toRationalClassOpKey = mkPreludeMiscIdUnique 193 -- Monad comprehensions -guardMIdKey, liftMIdKey, mzipIdKey :: Unique +guardMIdKey, mzipIdKey :: Unique guardMIdKey = mkPreludeMiscIdUnique 194 -liftMIdKey = mkPreludeMiscIdUnique 195 mzipIdKey = mkPreludeMiscIdUnique 196 -- GHCi @@ -2497,7 +2400,6 @@ proxyHashKey = mkPreludeMiscIdUnique 502 -- Used to make `Typeable` dictionaries mkTyConKey - , mkTrTypeKey , mkTrConKey , mkTrAppCheckedKey , mkTrFunKey @@ -2507,7 +2409,6 @@ mkTyConKey , typeRepIdKey :: Unique mkTyConKey = mkPreludeMiscIdUnique 503 -mkTrTypeKey = mkPreludeMiscIdUnique 504 mkTrConKey = mkPreludeMiscIdUnique 505 mkTrAppCheckedKey = mkPreludeMiscIdUnique 506 typeNatTypeRepKey = mkPreludeMiscIdUnique 507 @@ -2620,18 +2521,12 @@ integerFromNaturalIdKey , naturalQuotIdKey , naturalRemIdKey , naturalAndIdKey - , naturalAndNotIdKey , naturalOrIdKey , naturalXorIdKey , naturalTestBitIdKey , naturalBitIdKey , naturalGcdIdKey , naturalLcmIdKey - , naturalLog2IdKey - , naturalLogBaseWordIdKey - , naturalLogBaseIdKey - , naturalPowModIdKey - , naturalSizeInBaseIdKey , bignatEqIdKey , bignatCompareIdKey , bignatCompareWordIdKey @@ -2686,18 +2581,12 @@ naturalQuotRemIdKey = mkPreludeMiscIdUnique 669 naturalQuotIdKey = mkPreludeMiscIdUnique 670 naturalRemIdKey = mkPreludeMiscIdUnique 671 naturalAndIdKey = mkPreludeMiscIdUnique 672 -naturalAndNotIdKey = mkPreludeMiscIdUnique 673 naturalOrIdKey = mkPreludeMiscIdUnique 674 naturalXorIdKey = mkPreludeMiscIdUnique 675 naturalTestBitIdKey = mkPreludeMiscIdUnique 676 naturalBitIdKey = mkPreludeMiscIdUnique 677 naturalGcdIdKey = mkPreludeMiscIdUnique 678 naturalLcmIdKey = mkPreludeMiscIdUnique 679 -naturalLog2IdKey = mkPreludeMiscIdUnique 680 -naturalLogBaseWordIdKey = mkPreludeMiscIdUnique 681 -naturalLogBaseIdKey = mkPreludeMiscIdUnique 682 -naturalPowModIdKey = mkPreludeMiscIdUnique 683 -naturalSizeInBaseIdKey = mkPreludeMiscIdUnique 684 bignatEqIdKey = mkPreludeMiscIdUnique 691 bignatCompareIdKey = mkPreludeMiscIdUnique 692 ===================================== compiler/GHC/HsToCore/Foreign/Wasm.hs ===================================== @@ -224,6 +224,25 @@ especially since leaving all the boxing/unboxing business to C unifies the implementation of JSFFI imports and exports (rts_mkJSVal/rts_getJSVal). +We don't support unboxed FFI types like Int# etc. But we do support +one kind of unlifted FFI type for JSFFI import arguments: +ByteArray#/MutableByteArray#. The semantics is the same in C: the +pointer to the ByteArray# payload is passed instead of the ByteArray# +closure itself. This allows efficient zero-copy data exchange between +Haskell and JavaScript using unpinned ByteArray#, and the following +conditions must be met: + +- The JSFFI import itself must be a sync import marked as unsafe +- The JavaScript code must not re-enter Haskell when a ByteArray# is + passed as argument + +There's no magic in the handling of ByteArray#/MutableByteArray# +arguments. When generating C stub, we treat them like Ptr that points +to the payload, just without the rts_getPtr() unboxing call. After +lowering to C import, the backend takes care of adding the offset, see +add_shim in GHC.StgToCmm.Foreign and +Note [Unlifted boxed arguments to foreign calls]. + Now, each sync import calls a generated C function with a unique symbol. The C function uses rts_get* to unbox the arguments, call into JavaScript, then boxes the result with rts_mk* and returns it to @@ -517,8 +536,9 @@ importCStub sync cfun_name arg_tys res_ty js_src = CStub c_doc [] [] cfun_ret | res_ty `eqType` unitTy = cfun_call_import <> semi | otherwise = text "return" <+> cfun_call_import <> semi - cfun_make_arg arg_ty arg_val = - text ("rts_get" ++ ffiType arg_ty) <> parens arg_val + cfun_make_arg arg_ty arg_val + | isByteArrayPrimTy arg_ty = arg_val + | otherwise = text ("rts_get" ++ ffiType arg_ty) <> parens arg_val cfun_make_ret ret_val | res_ty `eqType` unitTy = ret_val | otherwise = @@ -543,7 +563,11 @@ importCStub sync cfun_name arg_tys res_ty js_src = CStub c_doc [] [] | res_ty `eqType` unitTy = text "void" | otherwise = text "HaskellObj" cfun_arg_list = - [text "HaskellObj" <+> char 'a' <> int n | n <- [1 .. length arg_tys]] + [ text (if isByteArrayPrimTy arg_ty then "HsPtr" else "HaskellObj") + <+> char 'a' + <> int n + | (arg_ty, n) <- zip arg_tys [1 ..] + ] cfun_args = case cfun_arg_list of [] -> text "void" _ -> hsep $ punctuate comma cfun_arg_list @@ -746,8 +770,18 @@ lookupGhcInternalTyCon m t = do n <- lookupOrig (mkGhcInternalModule m) (mkTcOcc t) dsLookupTyCon n +isByteArrayPrimTy :: Type -> Bool +isByteArrayPrimTy ty + | Just tc <- tyConAppTyCon_maybe ty, + tc == byteArrayPrimTyCon || tc == mutableByteArrayPrimTyCon = + True + | otherwise = + False + ffiType :: Type -> String -ffiType = occNameString . getOccName . fst . splitTyConApp +ffiType ty + | isByteArrayPrimTy ty = "Ptr" + | otherwise = occNameString $ getOccName $ tyConAppTyCon ty commonCDecls :: SDoc commonCDecls = ===================================== compiler/GHC/Parser/String.hs ===================================== @@ -19,6 +19,7 @@ import Data.Char (chr, ord) import qualified Data.Foldable1 as Foldable1 import qualified Data.List.NonEmpty as NonEmpty import Data.Maybe (listToMaybe, mapMaybe) +import GHC.Data.OrdList (fromOL, nilOL, snocOL) import GHC.Data.StringBuffer (StringBuffer) import qualified GHC.Data.StringBuffer as StringBuffer import GHC.Parser.CharClass ( @@ -167,16 +168,16 @@ collapseGaps = go [] -> panic "gap unexpectedly ended" resolveEscapes :: HasChar c => [c] -> Either (c, LexErr) [c] -resolveEscapes = go dlistEmpty +resolveEscapes = go nilOL where go !acc = \case - [] -> pure $ dlistToList acc + [] -> pure $ fromOL acc Char '\\' : Char '&' : cs -> go acc cs backslash@(Char '\\') : cs -> case resolveEscapeChar cs of - Right (esc, cs') -> go (acc `dlistSnoc` setChar esc backslash) cs' + Right (esc, cs') -> go (acc `snocOL` setChar esc backslash) cs' Left (c, e) -> Left (c, e) - c : cs -> go (acc `dlistSnoc` c) cs + c : cs -> go (acc `snocOL` c) cs -- ----------------------------------------------------------------------------- -- Escape characters @@ -420,17 +421,3 @@ It's more precisely defined with the following algorithm: * Lines with only whitespace characters 3. Calculate the longest prefix of whitespace shared by all lines in the remaining list -} - --- ----------------------------------------------------------------------------- --- DList - -newtype DList a = DList ([a] -> [a]) - -dlistEmpty :: DList a -dlistEmpty = DList id - -dlistToList :: DList a -> [a] -dlistToList (DList f) = f [] - -dlistSnoc :: DList a -> a -> DList a -dlistSnoc (DList f) x = DList (f . (x :)) ===================================== compiler/GHC/StgToCmm/InfoTableProv.hs ===================================== @@ -11,6 +11,7 @@ import GHC.IO (unsafePerformIO) #endif import Data.Char +import Data.Foldable import GHC.Prelude import GHC.Platform import GHC.Types.SrcLoc (pprUserRealSpan, srcSpanFile) @@ -18,6 +19,7 @@ import GHC.Types.Unique.DSM import GHC.Unit.Module import GHC.Utils.Outputable import GHC.Data.FastString (fastStringToShortText, unpackFS, LexicalFastString(..)) +import GHC.Data.OrdList (OrdList, nilOL, snocOL) import GHC.Cmm import GHC.Cmm.CLabel @@ -286,7 +288,7 @@ data CgInfoProvEnt = CgInfoProvEnt , ipeSrcSpan :: !StrTabOffset } -data StringTable = StringTable { stStrings :: DList ShortText +data StringTable = StringTable { stStrings :: !(OrdList ShortText) , stLength :: !Int , stLookup :: !(M.Map ShortText StrTabOffset) } @@ -295,7 +297,7 @@ type StrTabOffset = Word32 emptyStringTable :: StringTable emptyStringTable = - StringTable { stStrings = emptyDList + StringTable { stStrings = nilOL , stLength = 0 , stLookup = M.empty } @@ -303,7 +305,7 @@ emptyStringTable = getStringTableStrings :: StringTable -> BS.ByteString getStringTableStrings st = BSL.toStrict $ BSB.toLazyByteString - $ foldMap f $ dlistToList (stStrings st) + $ foldMap' f $ stStrings st where f x = BSB.shortByteString (ST.contents x) `mappend` BSB.word8 0 @@ -312,7 +314,7 @@ lookupStringTable str = state $ \st -> case M.lookup str (stLookup st) of Just off -> (off, st) Nothing -> - let !st' = st { stStrings = stStrings st `snoc` str + let !st' = st { stStrings = stStrings st `snocOL` str , stLength = stLength st + ST.byteLength str + 1 , stLookup = M.insert str res (stLookup st) } @@ -359,14 +361,3 @@ foreign import ccall unsafe "ZSTD_compressBound" defaultCompressionLevel :: Int defaultCompressionLevel = 3 - -newtype DList a = DList ([a] -> [a]) - -emptyDList :: DList a -emptyDList = DList id - -snoc :: DList a -> a -> DList a -snoc (DList f) x = DList (f . (x:)) - -dlistToList :: DList a -> [a] -dlistToList (DList f) = f [] ===================================== docs/users_guide/wasm.rst ===================================== @@ -265,7 +265,7 @@ backend’s JavaScript FFI, which we’ll now abbreviate as JSFFI. Marshalable types and ``JSVal`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -JSFFI supports all boxed marshalable foreign types in C FFI: +JSFFI supports all lifted marshalable foreign types in C FFI: - ``Bool`` - ``Char`` @@ -298,8 +298,14 @@ types in JSFFI. Some caveats to keep in mind: results in type errors, so keep this in mind. As for ``Int`` / ``Word``, they are 32-bit since the GHC wasm backend is based on ``wasm32`` . -- JSFFI doesn’t support unboxed foreign types like ``Int#``, - ``ByteArray#``, etc, even when ``UnliftedFFITypes`` is enabled. +- JSFFI doesn’t support unboxed foreign types like ``Int#``, even + when ``UnliftedFFITypes`` is enabled. The only supported unlifted + types are ``ByteArray#`` and ``MutableByteArray#``, they may only + be used as JSFFI import argument types, with the same semantics in + C FFI: the pointer to the payload is passed to JavaScript. Be + careful and avoid calling back into Haskell in such cases, + otherwise GC may occur and the pointer may be invalidated if it's + unpinned! In addition to the above types, JSFFI supports the ``JSVal`` type and its ``newtype``\ s as argument/result types. ``JSVal`` is defined in ===================================== hadrian/doc/flavours.md ===================================== @@ -242,6 +242,10 @@ The supported transformers are listed below: <td><code>ubsan</code></td> <td>Build all stage1+ C/C++ code with UndefinedBehaviorSanitizer support</td> </tr> + <tr> + <td><code>asan</code></td> + <td>Build all stage1+ C/C++ code with AddressSanitizer support</td> + </tr> <tr> <td><code>llvm</code></td> <td>Use GHC's LLVM backend (`-fllvm`) for all stage1+ compilation.</td> @@ -249,10 +253,6 @@ The supported transformers are listed below: <tr> <td><code>profiled_ghc</code></td> <td>Build the GHC executable with cost-centre profiling support. - It is recommended that you use this in conjunction with `no_dynamic_ghc` since - GHC does not support loading of profiled libraries with the - dynamic linker. You should use a flavour that builds profiling libs and rts, - i.e. not <code>quick</code>. <br> This flag adds cost centres with the -fprof-late flag.</td> </tr> <tr> @@ -274,6 +274,10 @@ The supported transformers are listed below: <td><code>text_simdutf</code></td> <td>Enable building the <code>text</code> package with <code>simdutf</code> support.</td> </tr> + <tr> + <td><code>with_profiled_libs</code></td> + <td>Enables building of stage1+ libraries and the RTS in profiled build ways (the opposite of <code>no_profiled_libs</code>).</td> + </tr> <tr> <td><code>no_profiled_libs</code></td> <td>Disables building of libraries in profiled build ways.</td> ===================================== hadrian/src/Flavour.hs ===================================== @@ -8,6 +8,7 @@ module Flavour , splitSections , enableThreadSanitizer , enableUBSan + , enableASan , enableLateCCS , enableHashUnitIds , enableDebugInfo, enableTickyGhc @@ -15,6 +16,7 @@ module Flavour , enableProfiledGhc , disableDynamicGhcPrograms , disableDynamicLibs + , enableProfiledLibs , disableProfiledLibs , enableLinting , enableHaddock @@ -56,12 +58,14 @@ flavourTransformers = M.fromList , "thread_sanitizer" =: enableThreadSanitizer False , "thread_sanitizer_cmm" =: enableThreadSanitizer True , "ubsan" =: enableUBSan + , "asan" =: enableASan , "llvm" =: viaLlvmBackend , "profiled_ghc" =: enableProfiledGhc , "no_dynamic_ghc" =: disableDynamicGhcPrograms , "no_dynamic_libs" =: disableDynamicLibs , "native_bignum" =: useNativeBignum , "text_simdutf" =: enableTextWithSIMDUTF + , "with_profiled_libs" =: enableProfiledLibs , "no_profiled_libs" =: disableProfiledLibs , "omit_pragmas" =: omitPragmas , "ipe" =: enableIPE @@ -169,6 +173,7 @@ enableDebugInfo :: Flavour -> Flavour enableDebugInfo = addArgs $ notStage0 ? mconcat [ builder (Ghc CompileHs) ? pure ["-g3"] , builder (Ghc CompileCWithGhc) ? pure ["-optc-g3"] + , builder (Ghc CompileCppWithGhc) ? pure ["-optcxx-g3"] , builder (Cc CompileC) ? arg "-g3" , builder (Cabal Setup) ? arg "--disable-library-stripping" , builder (Cabal Setup) ? arg "--disable-executable-stripping" @@ -303,33 +308,51 @@ enableUBSan = builder Testsuite ? arg "--config=have_ubsan=True" ] +-- | Build all stage1+ C/C++ code with AddressSanitizer support: +-- https://clang.llvm.org/docs/AddressSanitizer.html +enableASan :: Flavour -> Flavour +enableASan = + addArgs $ + notStage0 + ? mconcat + [ package rts + ? builder (Cabal Flags) + ? arg "+asan" + <> (needSharedLibSAN ? arg "+shared-libsan"), + builder (Ghc CompileHs) + ? arg "-optc-Og" + <> arg "-optc-fno-omit-frame-pointer" + <> arg "-optc-fsanitize=address", + builder (Ghc CompileCWithGhc) + ? ((not <$> input "**/Hash.c") ? arg "-optc-Og") + <> arg "-optc-fno-omit-frame-pointer" + <> arg "-optc-fsanitize=address", + builder (Ghc CompileCppWithGhc) + ? arg "-optcxx-Og" + <> arg "-optcxx-fno-omit-frame-pointer" + <> arg "-optcxx-fsanitize=address", + builder (Ghc LinkHs) + ? arg "-optc-Og" + <> arg "-optc-fno-omit-frame-pointer" + <> arg "-optc-fsanitize=address" + <> arg "-optl-fsanitize=address" + <> (needSharedLibSAN ? arg "-optl-shared-libsan"), + builder (Cc CompileC) + ? arg "-Og" + <> arg "-fno-omit-frame-pointer" + <> arg "-fsanitize=address", + builder Testsuite ? arg "--config=have_asan=True" + ] + -- | Use the LLVM backend in stages 1 and later. viaLlvmBackend :: Flavour -> Flavour viaLlvmBackend = addArgs $ notStage0 ? builder Ghc ? arg "-fllvm" --- | Build the GHC executable with profiling enabled in stages 2 and later. It --- is also recommended that you use this with @'dynamicGhcPrograms' = False@ --- since GHC does not support loading of profiled libraries with the --- dynamically-linker. +-- | Build the GHC executable with profiling enabled in stages 2 and +-- later. enableProfiledGhc :: Flavour -> Flavour enableProfiledGhc flavour = - enableLateCCS flavour - { rtsWays = do - ws <- rtsWays flavour - mconcat - [ pure ws - , buildingCompilerStage' (>= Stage2) ? pure (foldMap profiled_ways ws) - ] - , libraryWays = mconcat - [ libraryWays flavour - , buildingCompilerStage' (>= Stage2) ? pure (Set.singleton profiling) - ] - , ghcProfiled = (>= Stage2) - } - where - profiled_ways w - | wayUnit Dynamic w = Set.empty - | otherwise = Set.singleton (w <> profiling) + enableLateCCS $ enableProfiledLibs flavour { ghcProfiled = (>= Stage2) } -- | Disable 'dynamicGhcPrograms'. disableDynamicGhcPrograms :: Flavour -> Flavour @@ -346,6 +369,20 @@ disableDynamicLibs flavour = prune :: Ways -> Ways prune = fmap $ Set.filter (not . wayUnit Dynamic) +-- | Build libraries and the RTS in profiled ways (opposite of +-- 'disableProfiledLibs'). +enableProfiledLibs :: Flavour -> Flavour +enableProfiledLibs flavour = + flavour + { libraryWays = addProfilingWays $ libraryWays flavour, + rtsWays = addProfilingWays $ rtsWays flavour + } + where + addProfilingWays :: Ways -> Ways + addProfilingWays ways = do + ws <- ways + buildProfiled <- notStage0 + pure $ if buildProfiled then ws <> Set.map (<> profiling) ws else ws -- | Don't build libraries in profiled 'Way's. disableProfiledLibs :: Flavour -> Flavour ===================================== hadrian/src/Settings/Packages.hs ===================================== @@ -351,7 +351,7 @@ rtsPackageArgs = package rts ? do , Debug `wayUnit` way ? pure [ "-DDEBUG" , "-fno-omit-frame-pointer" , "-g3" - , "-O0" ] + , "-Og" ] -- Set the namespace for the rts fs functions , arg $ "-DFS_NAMESPACE=rts" ===================================== libraries/ghc-internal/src/GHC/Internal/IO/Handle.hs ===================================== @@ -480,7 +480,7 @@ hIsOpen handle = SemiClosedHandle -> return False _ -> return True --- | @'hIsOpen' hdl@ returns whether the handle is closed. +-- | @'hIsClosed' hdl@ returns whether the handle is closed. -- If the 'haType' of @hdl@ is 'ClosedHandle' this returns 'True' -- and 'False' otherwise. hIsClosed :: Handle -> IO Bool ===================================== rts/.ubsan-suppressions ===================================== @@ -1,3 +1,6 @@ +# libraries/bytestring/cbits/is-valid-utf8.c:66:14: runtime load of misaligned address 0x7ae45206f112 for type 'const uint64_t *' (aka 'const unsigned long *'), which requires 8 byte alignment +alignment:libraries/bytestring/cbits/is-valid-utf8.c + # libraries/text/cbits/measure_off.c:50:39: runtime left shift of 1 by 31 places cannot be represented in type 'int' shift-base:libraries/text/cbits/measure_off.c ===================================== rts/Task.c ===================================== @@ -183,6 +183,7 @@ freeTask (Task *task) stgFree(incall); } for (incall = task->spare_incalls; incall != NULL; incall = next) { + __ghc_asan_unpoison_memory_region(incall, sizeof(InCall)); next = incall->next; stgFree(incall); } @@ -252,6 +253,7 @@ newInCall (Task *task) if (task->spare_incalls != NULL) { incall = task->spare_incalls; + __ghc_asan_unpoison_memory_region(incall, sizeof(InCall)); task->spare_incalls = incall->next; task->n_spare_incalls--; } else { @@ -283,6 +285,7 @@ endInCall (Task *task) stgFree(incall); } else { incall->next = task->spare_incalls; + __ghc_asan_poison_memory_region(incall, sizeof(InCall)); task->spare_incalls = incall; task->n_spare_incalls++; } ===================================== rts/include/Stg.h ===================================== @@ -335,6 +335,7 @@ external prototype return neither of these types to workaround #11395. #include "stg/MachRegsForHost.h" #include "stg/Regs.h" #include "stg/Ticky.h" +#include "rts/ASANUtils.h" #include "rts/TSANUtils.h" #if IN_STG_CODE ===================================== rts/include/rts/ASANUtils.h ===================================== @@ -0,0 +1,33 @@ +#pragma once + +#if defined(__SANITIZE_ADDRESS__) +#define ASAN_ENABLED +#elif defined(__has_feature) +#if __has_feature(address_sanitizer) +#define ASAN_ENABLED +#endif +#endif + +#if defined(ASAN_ENABLED) +#include +#define USED_IF_ASAN +#else +#include +#define USED_IF_ASAN __attribute__((unused)) +#endif + +static inline void +__ghc_asan_poison_memory_region(void const volatile *addr USED_IF_ASAN, + size_t size USED_IF_ASAN) { +#if defined(ASAN_ENABLED) + __asan_poison_memory_region(addr, size); +#endif +} + +static inline void +__ghc_asan_unpoison_memory_region(void const volatile *addr USED_IF_ASAN, + size_t size USED_IF_ASAN) { +#if defined(ASAN_ENABLED) + __asan_unpoison_memory_region(addr, size); +#endif +} ===================================== rts/linker/InitFini.c ===================================== @@ -75,7 +75,7 @@ static void sortInitFiniList(struct InitFiniList **slist, enum SortOrder order) while (*last != NULL && (*last)->next != NULL) { struct InitFiniList *s0 = *last; struct InitFiniList *s1 = s0->next; - bool flip; + bool flip = false; switch (order) { case INCREASING: flip = s0->priority > s1->priority; break; case DECREASING: flip = s0->priority < s1->priority; break; ===================================== rts/rts.cabal ===================================== @@ -97,6 +97,12 @@ flag ubsan UndefinedBehaviorSanitizer. default: False manual: True +flag asan + description: + Link with -fsanitize=address, to be enabled when building with + AddressSanitizer. + default: False + manual: True flag shared-libsan description: Link with -shared-libsan, to guarantee only one copy of the @@ -216,6 +222,9 @@ library if flag(ubsan) ld-options: -fsanitize=undefined + if flag(asan) + ld-options: -fsanitize=address + if flag(shared-libsan) ld-options: -shared-libsan @@ -280,6 +289,7 @@ library -- ^ generated rts/ghc_ffi.h rts/Adjustor.h + rts/ASANUtils.h rts/ExecPage.h rts/BlockSignals.h rts/Bytecodes.h ===================================== rts/sm/MBlock.c ===================================== @@ -579,6 +579,8 @@ getMBlocks(uint32_t n) ret = getCommittedMBlocks(n); + __ghc_asan_unpoison_memory_region(ret, (W_)n * MBLOCK_SIZE); + debugTrace(DEBUG_gc, "allocated %d megablock(s) at %p",n,ret); mblocks_allocated += n; @@ -611,6 +613,8 @@ freeMBlocks(void *addr, uint32_t n) mblocks_allocated -= n; + __ghc_asan_poison_memory_region(addr, (W_)n * MBLOCK_SIZE); + decommitMBlocks(addr, n); } ===================================== rts/sm/Sanity.c ===================================== @@ -692,7 +692,7 @@ checkCompactObjects(bdescr *bd) ASSERT((W_)str == (W_)block + sizeof(StgCompactNFDataBlock)); StgWord totalW = 0; - StgCompactNFDataBlock *last; + StgCompactNFDataBlock *last = block; for ( ; block ; block = block->next) { last = block; ASSERT(block->owner == str); ===================================== testsuite/driver/testglobals.py ===================================== @@ -189,6 +189,9 @@ class TestConfig: # Are we running with UndefinedBehaviorSanitizer enabled? self.have_ubsan = False + # Are we running with AddressSanitizer enabled? + self.have_asan = False + # Do symbols use leading underscores? self.leading_underscore = False ===================================== testsuite/driver/testlib.py ===================================== @@ -1093,6 +1093,9 @@ def have_thread_sanitizer( ) -> bool: def have_ubsan( ) -> bool: return config.have_ubsan +def have_asan( ) -> bool: + return config.have_asan + def gcc_as_cmmp() -> bool: return config.cmm_cpp_is_gcc ===================================== testsuite/tests/ffi/should_run/all.T ===================================== @@ -192,6 +192,9 @@ test('rts_clearMemory', [ extra_ways(['g1', 'nursery_chunks', 'nonmoving', 'compacting_gc', 'sanity']), # On windows, nonmoving way fails with bad exit code (2816) when(opsys('mingw32'), fragile(23091)), + # For simplicity, ASAN poisoning/unpoisoning logic is omitted + # from rts_clearMemory implementation + when(have_asan(), skip), req_c, pre_cmd('$MAKE -s --no-print-directory rts_clearMemory_setup') ], # Same hack as ffi023 ===================================== testsuite/tests/jsffi/all.T ===================================== @@ -25,4 +25,6 @@ test('jsffion', [], compile_and_run, ['-optl-Wl,--export=main']) test('jsffisleep', [], compile_and_run, ['-optl-Wl,--export=testWouldBlock,--export=testLazySleep,--export=testThreadDelay,--export=testInterruptingSleep']) +test('bytearrayarg', [], compile_and_run, ['-optl-Wl,--export=main']) + test('textconv', [], compile_and_run, ['-optl-Wl,--export=main']) ===================================== testsuite/tests/jsffi/bytearrayarg.hs ===================================== @@ -0,0 +1,39 @@ +{-# LANGUAGE MagicHash #-} +{-# LANGUAGE UnboxedTuples #-} +{-# LANGUAGE UnliftedFFITypes #-} + +module Test where + +import GHC.Exts +import GHC.IO +import GHC.Word (Word8(W8#)) + +foreign import javascript unsafe "(() => { const u8 = new Uint8Array(__exports.memory.buffer, $1, 4); return (u8[0] === 0x12 && u8[1] === 0x34 && u8[2] === 0x56 && u8[3] === 0x78) ? 1 : 0; })()" + js_check_mba :: MutableByteArray# RealWorld -> IO Int + +foreign import javascript unsafe "(() => { const u8 = new Uint8Array(__exports.memory.buffer, $1, 4); return (u8[0] === 0x12 && u8[1] === 0x34 && u8[2] === 0x56 && u8[3] === 0x78) ? 1 : 0; })()" + js_check_ba :: ByteArray# -> IO Int + +foreign export javascript "main" + main :: IO () + +main :: IO () +main = + IO $ \s0 -> + case newPinnedByteArray# 4# s0 of + (# s1, mba# #) -> + case (0x12 :: Word8) of { W8# b0# -> + case (0x34 :: Word8) of { W8# b1# -> + case (0x56 :: Word8) of { W8# b2# -> + case (0x78 :: Word8) of { W8# b3# -> + let s2 = writeWord8Array# mba# 0# b0# s1 + s3 = writeWord8Array# mba# 1# b1# s2 + s4 = writeWord8Array# mba# 2# b2# s3 + s5 = writeWord8Array# mba# 3# b3# s4 + in case unIO (js_check_mba mba#) s5 of + (# s6, ok_mba #) -> case unsafeFreezeByteArray# mba# s6 of + (# s7, ba# #) -> case unIO (js_check_ba ba#) s7 of + (# s8, ok_ba #) -> case unIO (print ok_mba) s8 of + (# s9, _ #) -> case unIO (print ok_ba) s9 of + (# s10, _ #) -> (# s10, () #) + }}}} ===================================== testsuite/tests/jsffi/bytearrayarg.mjs ===================================== @@ -0,0 +1,4 @@ +export default async (__exports) => { + await __exports.main(); + process.exit(); +} ===================================== testsuite/tests/jsffi/bytearrayarg.stdout ===================================== @@ -0,0 +1,2 @@ +1 +1 ===================================== testsuite/tests/perf/should_run/all.T ===================================== @@ -420,6 +420,7 @@ test('T17949', [collect_stats('bytes allocated', 1), only_ways(['normal'])], com test('ByteCodeAsm', [ extra_run_opts('"' + config.libdir + '"') , js_broken(22261) + , when(arch('wasm32'), run_timeout_multiplier(10)) , collect_stats('bytes allocated', 10), ], compile_and_run, ===================================== testsuite/tests/rts/T18623/all.T ===================================== @@ -8,6 +8,8 @@ test('T18623', # Recent versions of osx report an error when running `ulimit -v` when(opsys('darwin'), skip), when(arch('powerpc64le'), skip), + # ASan can't allocate shadow memory + when(have_asan(), skip), cmd_prefix('ulimit -v ' + str(8 * 1024 ** 2) + ' && '), ignore_stdout], run_command, ===================================== testsuite/tests/rts/all.T ===================================== @@ -105,6 +105,8 @@ def remove_parenthesis(s): return re.sub(r'\s+\([^)]*\)', '', s) test('outofmem', [ when(opsys('darwin'), skip), + # ASan shadow memory allocation blows up + when(have_asan(), skip), # this is believed to cause other processes to die # that happen concurrently while the outofmem test # runs in CI. As such we'll need to disable it on View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/26339689beceb2a3483e2777341f259... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/26339689beceb2a3483e2777341f259... You're receiving this email because of your account on gitlab.haskell.org.