Hannes Siebenhandl pushed to branch wip/fendor/dep-json-link-nodes at Glasgow Haskell Compiler / GHC Commits: 8c2b7e98 by fendor at 2026-11-05T10:06:37+01:00 Add `-dep-json` flag to `ghc -M` Add an optional flag to `ghc -M` that allows to write the dependency information in JSON Lines format. The JSON Lines format enables us to stream the output of the dependency generation to a consumer. The output of `-dep-json` is streamed to a temporary file and then copied to the final destination to avoid partial writes. Each line contains one of two possible json objects: 1. A Home Unit Dependency * Contains the name of the home module, its unit, where it is located, whether a preprocessor is used, and all of its imports. Importantly, the imports include `-XExplicitLevelImports` information. * If requested via `-include-pkg-deps`, includes imports of external units, e.g. modules from `base`. * If requested via `-include-cpp-deps`, files included via CPP. 2. External Unit Dependency * Only if `-include-pkg-deps` is given * Contains unit-id, its package name and package id. --- This PR refactors the Makefile generation to use phases appropriately. We introduce an intermediate data structure that is generated based on the module graph nodes, and only then generate the Makefile rules. This allows the dep-json and dep-makefile generation to use the same dependency processing. - - - - - 44afdb4b by fendor at 2026-11-05T10:08:56+01:00 Add multiple home unit support to `ghc -M` Allows to process the module graph of multiple home units at once using the same syntax as `--make` and `--interactive`: ghc -M -unit @unitA -unit @unitB The semantics are identical to `--make` and `--interactive`. Each home unit can declare individually to use `-include-cpp-deps` and `-include-pkg-deps`. However, there can only be one output file, and `-dep-json` and `-dep-makefile` can only be provided at the top of the cli invocation. We introduce a new GHC API `doMkDepend` that allows to use multiple home units conveniently. The output of `-dep-json` and `-dep-makefile` does not change, since we were careful to choose a format that supports multiple home units from the start. Single home units are still accepted like before with no changes. - - - - - 084bd1ec by fendor at 2026-11-05T10:08:56+01:00 Deprecate `doMkDependHS` in favour of `doMkDepend` - - - - - 64f946a6 by fendor at 2026-11-05T10:20:17+01:00 Add support for streaming `ghc -M` to stdout Adds the flags `-dep-json-stdout` and `-dep-makefile-stdout` which streams dependency information line-by-line to stdout. Each dependency generation can be redirected to separate outputs. Providing any extra `-dep-{json,makefile}[-stdout]` disables the default generation of the `Makefile` file. Allows consumers to start processing the dependency information as it comes in and not waiting for the dependency generation to finish. `DepStream` abstracts over the possible output handles, which is either a file or the stdout handle. Writing to a file requires temporary file paths, thus `DepStream` must provide a way to perform clean up when dependency processing finished or when an error is thrown for any reason. We further decouple the rendering and writing of the result from the dependency generation, localising side effects and making the code slightly more readable. The testsuite now also handles `ghc -M` writing to stdout. We add a test for each combination of writing to stdout and file, except for writing both makefile and json. While it probably works, is really undefined behaviour. Closes #27937 - - - - - 370af6d5 by fendor at 2026-11-05T10:43:31+01:00 `ghc -M`: Add support for link nodes to json output `LinkNode`s are part of the module graph and indicate that a certain linking artefact is to be produced from other module graph nodes. We expose `LinkNode`s in the output of `ghc -M -dep-json`, allowing consumers to see when a linking artefact is to be produced. Only the JSON output format of `ghc -M` supports `LinkNode` as it is unclear how the Makefile output should be retrofitted to express the dependencies, and whether it would require a feature guard. Closes #28007 - - - - - 91 changed files: - + changelog.d/make-depends-deprecate-doMkDependHS - + changelog.d/make-depends-json - + changelog.d/make-depends-link-nodes - + changelog.d/make-depends-multiple-home-units - + changelog.d/make-depends-streaming-output - compiler/GHC/Driver/DynFlags.hs - compiler/GHC/Driver/MakeFile.hs - compiler/GHC/Driver/Session.hs - compiler/GHC/Driver/Session/Units.hs - compiler/GHC/Utils/Json.hs - + docs/users_guide/make-depends-json-schema-1_0.json - docs/users_guide/separate_compilation.rst - ghc/Main.hs - testsuite/driver/my_typing.py - testsuite/driver/testglobals.py - testsuite/driver/testlib.py - + testsuite/tests/driver/mkdepend/Makefile - + testsuite/tests/driver/mkdepend/README.md - + testsuite/tests/driver/mkdepend/T24384/A.hs - + testsuite/tests/driver/mkdepend/T24384/C.hs - + testsuite/tests/driver/mkdepend/T24384/C.hs-boot - + testsuite/tests/driver/mkdepend/T24384/D.hs - + testsuite/tests/driver/mkdepend/T24384/E.hs - + testsuite/tests/driver/mkdepend/T24384/dep-internal/DepInternal.hs - + testsuite/tests/driver/mkdepend/T24384/dep-internal/dep-internal.pkg - + testsuite/tests/driver/mkdepend/T24384/dep-public/DepPublic.hs - + testsuite/tests/driver/mkdepend/T24384/dep-public/dep-public.pkg - + testsuite/tests/driver/mkdepend/T24384/dep/Dep.hs - + testsuite/tests/driver/mkdepend/T24384/dep/dep.pkg - + testsuite/tests/driver/mkdepend/T24384/foo.h - + testsuite/tests/driver/mkdepend/T24384/lib/B.hs - + testsuite/tests/driver/mkdepend/T24384/lib/B.hs-boot - + testsuite/tests/driver/mkdepend/T24384/preproc.sh - + testsuite/tests/driver/mkdepend/T24384a.stdout - + testsuite/tests/driver/mkdepend/T24384b.stdout - + testsuite/tests/driver/mkdepend/T24384c.stdout - + testsuite/tests/driver/mkdepend/all.T - + testsuite/tests/driver/mkdepend/format_json.sh - + testsuite/tests/driver/mkdepend/format_makefile.sh - + testsuite/tests/driver/mkdepend/mhu01/a/A/Base.hs - + testsuite/tests/driver/mkdepend/mhu01/a/A/Base.hs-boot - + testsuite/tests/driver/mkdepend/mhu01/a/A/TH.hs - + testsuite/tests/driver/mkdepend/mhu01/a/Util.hs - + testsuite/tests/driver/mkdepend/mhu01/b/B/Normal.hs - + testsuite/tests/driver/mkdepend/mhu01/b/B/Splice.hs - + testsuite/tests/driver/mkdepend/mhu01/b/Util.hs - + testsuite/tests/driver/mkdepend/mhu01/c/C.hs - + testsuite/tests/driver/mkdepend/mhu01/unitA - + testsuite/tests/driver/mkdepend/mhu01/unitB - + testsuite/tests/driver/mkdepend/mhu01/unitC - + testsuite/tests/driver/mkdepend/mhu02/A.hs - + testsuite/tests/driver/mkdepend/mhu02/B.hs - + testsuite/tests/driver/mkdepend/mhu02/pp.sh - + testsuite/tests/driver/mkdepend/mhu02/unitA - + testsuite/tests/driver/mkdepend/mhu02/unitB - + testsuite/tests/driver/mkdepend/mhu03/pkg/A.hs - + testsuite/tests/driver/mkdepend/mhu03/pkg/pp.sh - + testsuite/tests/driver/mkdepend/mhu03/unitA - + testsuite/tests/driver/mkdepend/mhu04/app/Main.hs - + testsuite/tests/driver/mkdepend/mhu04/app/Utils.hs - + testsuite/tests/driver/mkdepend/mhu04/lib/Lib.hs - + testsuite/tests/driver/mkdepend/mhu04/lib/Lib/Internal.hs - + testsuite/tests/driver/mkdepend/mhu04/unitApp - + testsuite/tests/driver/mkdepend/mhu04/unitLib - + testsuite/tests/driver/mkdepend/mhu05/app/Main.hs - + testsuite/tests/driver/mkdepend/mhu05/lib/Lib.hs - + testsuite/tests/driver/mkdepend/mhu05/lib/Lib/Internal.hs - + testsuite/tests/driver/mkdepend/mhu05/test/Spec.hs - + testsuite/tests/driver/mkdepend/mhu05/unitApp - + testsuite/tests/driver/mkdepend/mhu05/unitLib - + testsuite/tests/driver/mkdepend/mhu05/unitTest - + testsuite/tests/driver/mkdepend/mkdepend01.dep-json - + testsuite/tests/driver/mkdepend/mkdepend01.dep-makefile - + testsuite/tests/driver/mkdepend/mkdepend01/A.hs - + testsuite/tests/driver/mkdepend/mkdepend02.dep-json - + testsuite/tests/driver/mkdepend/mkdepend02/Main.hs - + testsuite/tests/driver/mkdepend/mkdepend_bkp_01/bkp01.bkp - + testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-json - + testsuite/tests/driver/mkdepend/mkdepend_mhu01a.dep-makefile - + testsuite/tests/driver/mkdepend/mkdepend_mhu01b.stderr - + testsuite/tests/driver/mkdepend/mkdepend_mhu01c.stderr - + testsuite/tests/driver/mkdepend/mkdepend_mhu01d.dep-makefile - + testsuite/tests/driver/mkdepend/mkdepend_mhu01d.stderr - + testsuite/tests/driver/mkdepend/mkdepend_mhu02.dep-json - + testsuite/tests/driver/mkdepend/mkdepend_mhu03.dep-json - + testsuite/tests/driver/mkdepend/mkdepend_mhu04.dep-json - + testsuite/tests/driver/mkdepend/mkdepend_mhu04.dep-makefile - + testsuite/tests/driver/mkdepend/mkdepend_mhu05.dep-json - + testsuite/tests/driver/mkdepend/mkdepend_mhu05.dep-makefile - testsuite/tests/perf/compiler/large-project/all.T - + testsuite/tests/perf/compiler/large-project/large-project-mhu.sh The diff was not included because it is too large. View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/63b4f6c1945118d998d31881dc5a9d7... -- View it on GitLab: https://gitlab.haskell.org/ghc/ghc/-/compare/63b4f6c1945118d998d31881dc5a9d7... You're receiving this email because of your account on gitlab.haskell.org. Manage all notifications: https://gitlab.haskell.org/-/profile/notifications | Help: https://gitlab.haskell.org/help