[Git][ghc/ghc][master] 5 commits: wasm: reformat dyld source code
Marge Bot pushed to branch master at Glasgow Haskell Compiler / GHC Commits: f6961b02 by Cheng Shao at 2025-11-01T00:08:01+01:00 wasm: reformat dyld source code This commit reformats dyld source code with prettier, to avoid introducing unnecessary diffs in subsequent patches when they're formatted before committing. - - - - - 0c9032a0 by Cheng Shao at 2025-11-01T00:08:01+01:00 wasm: simplify _initialize logic in dyld This commit simplifies how we _initialize a wasm shared library in dyld and removes special treatment for libc.so, see added comment for detailed explanation. - - - - - ec1b40bd by Cheng Shao at 2025-11-01T00:08:01+01:00 wasm: support running dyld fully client side in the browser This commit refactors the wasm dyld script so that it can be used to load and run wasm shared libraries fully client-side in the browser without needing a wasm32-wasi-ghci backend: - A new `DyLDBrowserHost` class is exported, which runs in the browser and uses the in-memory vfs without any RPC calls. This meant to be used to create a `rpc` object for the fully client side use cases. - The exported `main` function now can be used to load user-specified shared libraries, and the user can use the returned `DyLD` instance to run their own exported Haskell functions. - The in-browser wasi implementation is switched to https://github.com/haskell-wasm/browser_wasi_shim for bugfixes and major performance improvements not landed upstream yet. - When being run by deno, it now correctly switches to non-nodejs code paths, so it's more convenient to test dyld logic with deno. See added comments for details, as well as the added `playground001` test case for an example of using it to build an in-browser Haskell playground. - - - - - 8f3e481f by Cheng Shao at 2025-11-01T00:08:01+01:00 testsuite: add playground001 to test haskell playground This commit adds the playground001 test case to test the haskell playground in browser, see comments for details. - - - - - af40606a by Cheng Shao at 2025-11-01T00:08:04+01:00 Revert "testsuite: add T26431 test case" This reverts commit 695036686f8c6d78611edf3ed627608d94def6b7. T26431 is now retired, wasm ghc internal-interpreter logic is tested by playground001. - - - - - 10 changed files: - + testsuite/tests/ghc-api-browser/README.md - + testsuite/tests/ghc-api-browser/all.T - + testsuite/tests/ghc-api-browser/index.html - + testsuite/tests/ghc-api-browser/playground001.hs - + testsuite/tests/ghc-api-browser/playground001.js - + testsuite/tests/ghc-api-browser/playground001.sh - testsuite/tests/ghci-wasm/T26431.stdout → testsuite/tests/ghc-api-browser/playground001.stdout - − testsuite/tests/ghci-wasm/T26431.hs - testsuite/tests/ghci-wasm/all.T - utils/jsffi/dyld.mjs Changes: ===================================== testsuite/tests/ghc-api-browser/README.md ===================================== @@ -0,0 +1,124 @@ +# The Haskell playground browser test + +This directory contains the `playground001` test, which builds a fully +client side Haskell playground in the browser, then runs a +puppeteer-based test to actually interpret a Haskell program in a +headless browser. + +## Headless testing + +`playground001` is tested in GHC CI. To test it locally, first ensure +you've set up the latest +[`ghc-wasm-meta`](https://gitlab.haskell.org/haskell-wasm/ghc-wasm-meta) +toolchain and sourced the `~/.ghc-wasm/env` script, so the right +`node` with the right pre-installed libraries are used. Additionally, +you need to install latest Firefox and: + +```sh +export FIREFOX_LAUNCH_OPTS='{"browser":"firefox","executablePath":"/usr/bin/firefox"}'` +``` + +Or on macOS: + +```sh +export FIREFOX_LAUNCH_OPTS='{"browser":"firefox","executablePath":"/Applications/Firefox.app/Contents/MacOS/firefox"}' +``` + +Without `FIREFOX_LAUNCH_OPTS`, `playground001` is skipped. + +It's possible to test against Chrome as well, the +[`playground001.js`](./playground001.js) test driver doesn't assume +anything Firefox-specific, it just takes the +[`puppeteer.launch`](https://pptr.dev/api/puppeteer.puppeteernode.launch) +options as JSON passed via command line. + +`playground001` works on latest versions of Firefox/Chrome/Safari. + +## Manual testing + +The simplest way to build the playground manually and run it in a +browser tab is to test it once with `--only=playground001 +--keep-test-files` passed to Hadrian, then you can find the temporary +directory containing [`index.html`](./index.html), `rootfs.tar.zst` +etc, then fire up a dev web server and load it. + +Additionally, you can build the playground in tree without invoking +the GHC testsuite. Just build GHC with the wasm target first, then +copy `utils/jsffi/*.mjs` here and run +[`./playground001.sh`](./playground001.sh) script. You need to set +`TEST_CC` to the path of `wasm32-wasi-clang` and `TEST_HC` to the path +of `wasm32-wasi-ghc`, that's it. + +## Customized Haskell playground + +You may want to build a customized Haskell playground that uses GHC +API to interpret Haskell code with custom packages, here are some tips +to get started: + +- Read the code in this directory and figure out how `playground001` + itself works. +- [`./playground001.sh`](./playground001.sh) can be used as a basis to + write your own build/test script. + +You don't need to read the full `dyld.mjs` script. The user-facing +things that are relevant to the playground use case are: + +- `export class DyLDBrowserHost`: it is the `rpc` object required when + calling `main`. You need to pass `stdout`/`stderr` callbacks to + write each line of stdout/stderr, as well as a `rootfs` object that + represents an in-memory vfs containing the shared libraries to load. +- `export async function main`: it eventually returns a `DyLD` object + that can be used like `await + dyld.exportFuncs.myExportedHaskellFunc(js_foo, js_bar)` to invoke + your exported Haskell function. + +Check the source code of [`index.html`](./index.html) and cross +reference [`playground001.hs`](./playground001.hs) for the example of +how they are used. + +The `rootfs` object is a +[`PreopenDirectory`](https://github.com/haskell-wasm/browser_wasi_shim/blob/master/src/fs_mem.ts) +object in the +[`browser_wasi_shim`](https://github.com/haskell-wasm/browser_wasi_shim) +library. The Haskell playground needs a complex vfs containing many +files (shared libraries, interface files, package databases, etc), so +to speed things up, the whole vfs is compressed into a +`rootfs.tar.zst` archive, then that archive is extracted using +[`bsdtar-wasm`](https://github.com/haskell-wasm/bsdtar-wasm). + +You don't need to read the source code of `browser_wasi_shim`; you can +simply paste and adapt the relevant code snippet in +[`index.html`](./index.html) to create the right `rootfs` object from +a tarball. + +The main concern is what do you need to pack into `rootfs.tar.zst`. +For `playground001`, it contains: + +- `/tmp/clib`: the C/C++ shared libraries +- `/tmp/hslib/lib`: the GHC libdir +- `/tmp/libplayground001.so`: the main shared library to start loading + that exports `myMain` + +You can read [`./playground001.sh`](./playground001.sh) to figure out +the details of how I prepare `rootfs.tar.zst` and trim unneeded files +to minimize the tarball size. + +There are multiple possible ways to install third-party packages in +the playground: + +- Start from a `wasm32-wasi-ghc` installation, use `wasm32-wasi-cabal + v1-install --global` to install everything to the global package + database. In theory this is the simplest way, though I haven't tried + it myself and it's unclear to what extent do `v1` commands work + these days. +- Use default nix-style installation, then package the cabal store and + `dist-newstyle` directories into `rootfs.tar.zst`, and pass the + right package database flags when calling GHC API. + +Note that cabal built packages are not relocatable! So things will +break if you build them at a host location and then package into a +different absolute path into the rootfs, keep this in mind. + +If you have any difficulties, you're welcome to the [Haskell +Wasm](https://matrix.to/#/#haskell.wasm:matrix.org) matrix room for +community support. ===================================== testsuite/tests/ghc-api-browser/all.T ===================================== @@ -0,0 +1,52 @@ +# makefile_test/run_command is skipped when config.target_wrapper is +# not None, see test_common_work in testsuite/driver/testlib.py. for +# now just use this workaround to run custom test script here; ideally +# we'd fix test failures elsewhere and enable +# makefile_test/run_command for cross targets some day. +async def stub_run_command(name, way, cmd): + return await run_command(name, way, cmd) + + +# config.target_wrapper is prepended when running any command when +# testing a cross target, see simple_run in +# testsuite/driver/testlib.py. this is problematic when running a host +# test script. for now do this override; ideally we'd have clear +# host/target distinction for command invocations in the testsuite +# driver instead of just a command string. +def override_target_wrapper(name, opts): + opts.target_wrapper = "" + + +setTestOpts( + [ + unless(arch("wasm32"), skip), + override_target_wrapper, + high_memory_usage, + ignore_stderr, + only_ways(["dyn"]), + extra_ways(["dyn"]), + ] +) + + +test( + "playground001", + [ + # pretty heavyweight, just test one browser for now. + unless("FIREFOX_LAUNCH_OPTS" in ghc_env, skip), + extra_files( + [ + "../../../.gitlab/hello.hs", + "../../../utils/jsffi/dyld.mjs", + "../../../utils/jsffi/post-link.mjs", + "../../../utils/jsffi/prelude.mjs", + "index.html", + "playground001.hs", + "playground001.js", + "playground001.sh", + ] + ), + ], + stub_run_command, + ['./playground001.sh "$FIREFOX_LAUNCH_OPTS"'], +) ===================================== testsuite/tests/ghc-api-browser/index.html ===================================== @@ -0,0 +1,234 @@ +<!DOCTYPE html> +<html lang="en"> + <head> + <meta charset="utf-8" /> + <meta name="viewport" content="width=device-width, initial-scale=1" /> + <title>ghc-in-browser</title> + https://cdn.jsdelivr.net/npm/modern-normalize/modern-normalize.min.css" + /> + <style> + html, + body { + height: 100%; + } + body { + margin: 0; + font-family: system-ui, -apple-system, Segoe UI, Roboto, sans-serif; + background: #0f172a; + color: #e5e7eb; + } + .app { + height: 100vh; + display: grid; + gap: 0.5rem; + padding: 0.5rem; + } + @media (min-width: 800px) { + .app { + grid-template-columns: 1fr 1fr; + } + } + @media (max-width: 799.98px) { + .app { + grid-template-rows: 1fr 1fr; + } + } + .pane { + background: #111827; + border: 1px solid #1f2937; + border-radius: 12px; + display: flex; + flex-direction: column; + min-height: 0; + } + header { + padding: 0.5rem 0.75rem; + border-bottom: 1px solid #1f2937; + font-weight: 600; + } + #editor { + flex: 1; + min-height: 0; + } + .right { + padding: 0.6rem; + gap: 0.6rem; + } + .controls { + display: flex; + gap: 0.5rem; + flex-wrap: wrap; + margin-bottom: 0.4rem; + } + .controls input[type="text"] { + flex: 1; + min-width: 200px; + background: #0b1020; + color: #e5e7eb; + border: 1px solid #223; + border-radius: 8px; + padding: 0.55rem; + } + .controls button { + background: #22c55e; + border: none; + border-radius: 8px; + padding: 0.55rem 0.85rem; + font-weight: 600; + cursor: pointer; + } + .outputs { + display: block; + } + .outputs .label { + font-size: 0.85rem; + opacity: 0.8; + margin: 0.35rem 0; + } + .outputs textarea { + display: block; + width: 100%; + min-height: 30vh; + background: #0b1020; + color: #d1fae5; + border: 1px solid #223; + border-radius: 8px; + padding: 0.6rem; + resize: vertical; + } + .stderr { + color: #fee2e2; + } + </style> + + <script async type="module"> + import * as monaco from "https://cdn.jsdelivr.net/npm/monaco-editor/+esm"; + import { + ConsoleStdout, + File, + OpenFile, + PreopenDirectory, + WASI, + } from "https://esm.sh/gh/haskell-wasm/browser_wasi_shim"; + import { DyLDBrowserHost, main } from "./dyld.mjs"; + + const rootfs = new PreopenDirectory("/", []); + + const bsdtar_wasi = new WASI( + ["bsdtar.wasm", "-x"], + [], + [ + new OpenFile(new File(new Uint8Array(), { readonly: true })), + ConsoleStdout.lineBuffered((msg) => console.info(msg)), + ConsoleStdout.lineBuffered((msg) => console.warn(msg)), + rootfs, + ], + { debug: false } + ); + + const [{ instance }, rootfs_bytes] = await Promise.all([ + WebAssembly.instantiateStreaming( + fetch("https://haskell-wasm.github.io/bsdtar-wasm/bsdtar.wasm"), + { wasi_snapshot_preview1: bsdtar_wasi.wasiImport } + ), + fetch("./rootfs.tar.zst").then((r) => r.bytes()), + ]); + + bsdtar_wasi.fds[0] = new OpenFile( + new File(rootfs_bytes, { readonly: true }) + ); + bsdtar_wasi.start(instance); + + if (document.readyState === "loading") { + await new Promise((res) => + document.addEventListener("DOMContentLoaded", res, { once: true }) + ); + } + + window.editor = monaco.editor.create(document.getElementById("editor"), { + value: 'main :: IO ()\nmain = putStrLn "Hello, Haskell!"\n', + language: "haskell", + automaticLayout: true, + minimap: { enabled: false }, + theme: "vs-dark", + fontSize: 14, + }); + + const dyld = await main({ + rpc: new DyLDBrowserHost({ + rootfs, + stdout: (msg) => { + document.getElementById("stdout").value += `${msg}\n`; + }, + stderr: (msg) => { + document.getElementById("stderr").value += `${msg}\n`; + }, + }), + searchDirs: [ + "/tmp/clib", + "/tmp/hslib/lib/wasm32-wasi-ghc-9.15.20251024", + ], + mainSoPath: "/tmp/libplayground001.so", + args: ["libplayground001.so", "+RTS", "-c", "-RTS"], + isIserv: false, + }); + const main_func = await dyld.exportFuncs.myMain("/tmp/hslib/lib"); + + document.getElementById("runBtn").addEventListener("click", async () => { + document.getElementById("runBtn").disabled = true; + + try { + document.getElementById("stdout").value = ""; + document.getElementById("stderr").value = ""; + + await main_func( + document.getElementById("ghcArgs").value, + editor.getValue() + ); + } finally { + document.getElementById("runBtn").disabled = false; + } + }); + + document.getElementById("runBtn").disabled = false; + </script> + </head> + <body> + <div class="app"> + <section class="pane"> + <header>Haskell Source</header> + <div id="editor"></div> + </section> + + <section class="pane right"> + <header>Controls / Output</header> + <div class="controls"> + + <button id="runBtn" disabled="true">Run</button> + </div> + <div class="outputs"> + <div class="label">stdout</div> +