Hello GHC devs, I took the opportunity to give Hadrian a test-run to see whether it could live up to the big promise of delivering a "more scalable, faster" system than the current GNU Make based system. Unfortunately, my preliminary results don't back this claim, and actually make Hadrian appear to be significantly slower. ---- Here's the summary of the results: | Hadrian | GNU Make | +------------------------+-----------+-----------+ | Compiling `hadrian` | 5m25s | 0 | | (one-time setup) | | | +------------------------+-----------+-----------+ | build "all" at -j8 | 38m | 33m | +------------------------+-----------+-----------+ | no-op build at -j8 | 10.977s | 3.258s | +------------------------+-----------+-----------+ | "clean" | 21s | 51s | +------------------------+-----------+-----------+ So, Hadrian is ~5 minutes than GNU Make (or even ~10 minutes if you also count the one-time setup cost). And what I personally consider a bit annoying is that it's ~3 times slower detecting; i.e. you have to wait 11s for Hadrian to detect there's nothing to be done which compared to GNU Make (which currently needlessly re-runs Sphinx; so it could be even faster!) is very noticeable to me. There's a silver-lining though, deleting files is the part which is a lot more costly in the GNU Make system currently since artifacts are spread over several (scroll to the end of this email) subfolders there. Whereas Hadrian did something we should have done for the GNU Make system as well (and probably would have done sooner or later anyway in order to support the srcdir!=buildir scheme that people are used to from GNU Autotools projects); Hadrian places build-artifacts into a few top-level folders, and so cleaning up is trivial and requires to unlink only a few folders from the filesystem. ---- At the very least, I'd expect Hadrian to be as fast as the GNU Make system (and ideally beat it, not the least as this was besides maintainability its big promise), but so far it doesn't seem to deliver that promise for me. It could easily be that I'm comparing apples to oranges here or that I've otherwise overlooked something, so let me describe how I came to this conclusion: I tried this on an reasonably idle Linux workstation with an Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz CPU, and with 32GiB RAM (i.e. the filesystem content was well-cached into memory; NB: a ghc source tree + compiled artifacts takes up about 4GiB on the filesystem). I've started from a fresh Git clone, i.e. git clone --recursive git://git.haskell.org/ghc.git followed by ./boot ./configure At this point, we're at the common point from which both the Hadrian and the GNU Make build-system would start diverging: For the Hadrian build-system, we need to pay for a one-time setup, since we need to build the `hadrian` executable (which requires to build the in-tree lib:Cabal as an in-place library): $ time ./hadrian/build.sh --help ... real 5m25.992s user 6m19.196s sys 0m6.079s I'm not too worried about this part, as there's a few tricks by which we could likely bring that down to about 2 minutes or so, and we mostly pay this setup-cost, when lib:Cabal and/or `hadrian` changes and requires to be recompiled. Now, after having made sure that the `hadrian` executable is fresh, I started the actual build: $ time ./hadrian/build.sh -j8 ... shakeArgsWith 0.000s 0% Function shake 0.178s 0% Database read 0.000s 0% With database 0.000s 0% Running rules 2338.398s 99% ========================= Pool finished (5261 threads, 8 max) 0.002s 0% Lint checking 0.111s 0% Total 2338.690s 100% Build completed in 38:59m real 38m59.626s user 219m7.421s sys 11m7.584s then I immediately re-issued the same command to test how long it takes to perform a no-op build: $ time ./hadrian/build.sh -j8 Up to date Up to date shakeArgsWith 0.000s 0% Function shake 0.183s 1% Database read 0.144s 1% With database 0.241s 2% Running rules 9.379s 93% ========================= Pool finished (4165 threads, 8 max) 0.004s 0% Lint checking 0.101s 1% Total 10.051s 100% Build completed in 0:11m real 0m10.977s user 0m19.399s sys 0m2.443s Same without -j8: $ time ./hadrian/build.sh Up to date Up to date shakeArgsWith 0.000s 0% Function shake 0.175s 2% Database read 0.123s 1% With database 0.197s 2% Running rules 7.710s 92% ========================= Pool finished (1 threads, 1 max) 0.003s 0% Lint checking 0.084s 1% Total 8.293s 100% Build completed in 0:09m real 0m9.196s user 0m8.656s sys 0m0.724s And finally clean it again: $ sync $ time ./hadrian/build.sh clean Up to date Up to date | Remove directory _build/stage0 | Remove directory _build/stage1 | Remove directory _build/stage2 | Remove directory _build/stage3 | Remove directory inplace/bin | Remove directory inplace/lib | Remove directory sdistprep | Remove Hadrian files... | Remove directory _build/generated | Done. shakeArgsWith 0.000s 0% Function shake 0.175s 0% Database read 0.114s 0% With database 0.205s 0% Running rules 20.037s 97% ======================== Pool finished (1 threads, 1 max) 0.002s 0% Lint checking 0.006s 0% Total 20.540s 100% Build completed in 0:21m real 0m21.426s user 0m1.415s sys 0m1.045s ---- Running the full build via our rusty GNU Make system: $ time make V=0 -j8 real 33m30.801s user 157m49.520s sys 6m49.289s A no-op build: $ time make V=0 -j8 ===--- building phase 0 make --no-print-directory -f ghc.mk phase=0 phase_0_builds make[1]: Nothing to be done for 'phase_0_builds'. ===--- building phase 1 make --no-print-directory -f ghc.mk phase=1 phase_1_builds make[1]: Nothing to be done for 'phase_1_builds'. ===--- building final phase make --no-print-directory -f ghc.mk phase=final all make -C utils/haddock/doc html SPHINX_BUILD=/usr/bin/sphinx-build /usr/bin/sphinx-build -b html . .build-html Running Sphinx v1.3.6 loading translations [en]... done loading pickled environment... done building [mo]: targets for 0 po files that are out of date building [html]: targets for 0 source files that are out of date updating environment: 0 added, 0 changed, 0 removed looking for now-outdated files... none found no targets are out of date. build succeeded. cp -R utils/haddock/doc/.build-html utils/haddock/doc/haddock real 0m3.258s user 0m3.134s sys 0m0.283s And finally `make clean`: $ sync $ time make clean make --no-print-directory -f ghc.mk clean CLEANING=YES "rm" -rf inplace/bin inplace/lib "rm" -rf utils/touchy/dist "rm" -rf inplace/lib/bin/touchy "rm" -rf utils/unlit/dist "rm" -rf inplace/lib/bin/unlit "rm" -rf utils/unlit/dist-install "rm" -rf utils/hp2ps/dist "rm" -rf inplace/bin/hp2ps "rm" -rf inplace/lib/bin/hp2ps "rm" -rf utils/hp2ps/dist-install "rm" -rf driver/split/dist inplace/lib/bin/ghc-split "rm" -rf utils/genprimopcode/dist "rm" -rf inplace/bin/genprimopcode "rm" -rf libffi/build libffi/stamp.ffi.static-shared.configure libffi/stamp.ffi.static-shared.build libffi/stamp.ffi.static-shared.install libffi/dist-install "rm" -rf utils/deriveConstants/dist "rm" -rf inplace/bin/deriveConstants "rm" -rf includes/ghcautoconf.h includes/ghcplatform.h includes/ghcversion.h "rm" -rf rts/dist "rm" -rf bindisttest/"install dir" bindisttest/HelloWorld bindisttest/HelloWorld.o bindisttest/HelloWorld.hi bindisttest/output "rm" -rf utils/genapply/dist "rm" -rf inplace/bin/genapply "rm" -rf libraries/integer-gmp/include/ghc-gmp.h libraries/integer-gmp/gmp/config.mk libraries/integer-gmp/gmp/libgmp.a libraries/integer-gmp/gmp/gmp.h libraries/integer-gmp/gmp/gmpbuild libraries/integer-gmp/gmp/gmp-6.1.2 "rm" -rf libraries/integer-gmp/gmp/objs "rm" -rf libraries/integer-gmp/gmp/gmpbuild "rm" -rf utils/haddock/dist "rm" -rf inplace/bin/haddock "rm" -rf inplace/lib/bin/haddock "rm" -rf compiler/stage1 "rm" -rf compiler/stage2 "rm" -rf compiler/stage3 "rm" -rf utils/hsc2hs/dist "rm" -rf inplace/bin/hsc2hs "rm" -rf inplace/lib/bin/hsc2hs "rm" -rf utils/hsc2hs/dist-install "rm" -rf utils/ghc-pkg/dist "rm" -rf inplace/bin/ghc-pkg "rm" -rf inplace/lib/bin/ghc-pkg "rm" -rf utils/ghc-pkg/dist-install "rm" -rf utils/ghctags/dist-install "rm" -rf inplace/bin/ghctags "rm" -rf utils/check-api-annotations/dist-install "rm" -rf inplace/bin/check-api-annotations "rm" -rf utils/check-ppr/dist-install "rm" -rf inplace/bin/check-ppr "rm" -rf utils/ghc-cabal/dist bootstrapping "rm" -rf utils/ghc-cabal/dist-install "rm" -rf utils/hpc/dist-install "rm" -rf inplace/bin/hpc "rm" -rf inplace/lib/bin/hpc "rm" -rf utils/runghc/dist-install "rm" -rf inplace/bin/runghc "rm" -rf inplace/lib/bin/runghc "rm" -rf ghc/stage1 "rm" -rf inplace/bin/ghc-stage1 "rm" -rf inplace/lib/bin/ghc-stage1 "rm" -rf ghc/stage2 "rm" -rf inplace/bin/ghc-stage2 "rm" -rf inplace/lib/bin/ghc-stage2 "rm" -rf ghc/stage3 "rm" -rf docs/users_guide/.doctrees-html/ docs/users_guide/.doctrees-pdf/ docs/users_guide/build-html/ docs/users_guide/build-pdf/ docs/users_guide/users_guide.pdf "rm" -rf docs/users_guide/.doctrees-man/ docs/users_guide/build-man/ "rm" -rf utils/count_lines/dist inplace/bin/count_lines "rm" -rf utils/compare_sizes/dist-install "rm" -rf iserv/stage2 "rm" -rf inplace/lib/bin/ghc-iserv "rm" -rf iserv/stage2_p "rm" -rf inplace/lib/bin/ghc-iserv-prof "rm" -rf iserv/stage2_dyn "rm" -rf inplace/lib/bin/ghc-iserv-dyn "rm" -f libraries/integer-gmp/include/HsIntegerGmp.h libraries/base/include/EventConfig.h mk/config.mk.old mk/project.mk.old compiler/ghc.cabal.old includes/GHCConstants.h includes/DerivedConstants.h includes/ghcautoconf.h includes/ghcplatform.h includes/ghcversion.h utils/ghc-pkg/Version.hs compiler/prelude/primops.txt "rm" -rf includes/dist-derivedconstants "rm" -rf inplace/bin "rm" -rf inplace/lib "rm" -rf libraries/bootstrapping.conf "rm" -f mk/are-validating.mk "rm" -rf libraries/ghc-boot-th/dist-install "rm" -rf libraries/ghc-boot/dist-install "rm" -rf libraries/ghci/dist-install "rm" -rf libraries/base/dist-install "rm" -rf libraries/ghc-prim/dist-install "rm" -rf libraries/integer-gmp/dist-install "rm" -rf libraries/integer-simple/dist-install "rm" -rf libraries/template-haskell/dist-install "rm" -rf libraries/array/dist-install "rm" -rf libraries/binary/dist-install "rm" -rf libraries/bytestring/dist-install "rm" -rf libraries/Cabal/Cabal/dist-install "rm" -rf libraries/ghc-compact/dist-install "rm" -rf libraries/containers/dist-install "rm" -rf libraries/deepseq/dist-install "rm" -rf libraries/directory/dist-install "rm" -rf libraries/filepath/dist-install "rm" -rf libraries/haskeline/dist-install "rm" -rf libraries/hpc/dist-install "rm" -rf libraries/mtl/dist-install "rm" -rf libraries/parsec/dist-install "rm" -rf libraries/pretty/dist-install "rm" -rf libraries/process/dist-install "rm" -rf libraries/terminfo/dist-install "rm" -rf libraries/text/dist-install "rm" -rf libraries/time/dist-install "rm" -rf libraries/transformers/dist-install "rm" -rf libraries/unix/dist-install "rm" -rf libraries/Win32/dist-install "rm" -rf libraries/xhtml/dist-install "rm" -rf libraries/parallel/dist-install "rm" -rf libraries/stm/dist-install "rm" -rf libraries/random/dist-install "rm" -rf libraries/primitive/dist-install "rm" -rf libraries/vector/dist-install "rm" -rf libraries/dph/dph-base/dist-install "rm" -rf libraries/dph/dph-prim-interface/dist-install "rm" -rf libraries/dph/dph-prim-seq/dist-install "rm" -rf libraries/dph/dph-prim-par/dist-install "rm" -rf libraries/dph/dph-lifted-base/dist-install "rm" -rf libraries/dph/dph-lifted-boxed/dist-install "rm" -rf libraries/dph/dph-lifted-copy/dist-install "rm" -rf libraries/dph/dph-lifted-vseg/dist-install "rm" -rf libraries/ghc-boot-th/dist-boot "rm" -rf libraries/ghc-boot/dist-boot "rm" -rf libraries/ghci/dist-boot "rm" -rf libraries/base/dist-boot "rm" -rf libraries/ghc-prim/dist-boot "rm" -rf libraries/integer-gmp/dist-boot "rm" -rf libraries/integer-simple/dist-boot "rm" -rf libraries/template-haskell/dist-boot "rm" -rf libraries/array/dist-boot "rm" -rf libraries/binary/dist-boot "rm" -rf libraries/bytestring/dist-boot "rm" -rf libraries/Cabal/Cabal/dist-boot "rm" -rf libraries/ghc-compact/dist-boot "rm" -rf libraries/containers/dist-boot "rm" -rf libraries/deepseq/dist-boot "rm" -rf libraries/directory/dist-boot "rm" -rf libraries/filepath/dist-boot "rm" -rf libraries/haskeline/dist-boot "rm" -rf libraries/hpc/dist-boot "rm" -rf libraries/mtl/dist-boot "rm" -rf libraries/parsec/dist-boot "rm" -rf libraries/pretty/dist-boot "rm" -rf libraries/process/dist-boot "rm" -rf libraries/terminfo/dist-boot "rm" -rf libraries/text/dist-boot "rm" -rf libraries/time/dist-boot "rm" -rf libraries/transformers/dist-boot "rm" -rf libraries/unix/dist-boot "rm" -rf libraries/Win32/dist-boot "rm" -rf libraries/xhtml/dist-boot "rm" -rf libraries/parallel/dist-boot "rm" -rf libraries/stm/dist-boot "rm" -rf libraries/random/dist-boot "rm" -rf libraries/primitive/dist-boot "rm" -rf libraries/vector/dist-boot "rm" -rf libraries/dph/dph-base/dist-boot "rm" -rf libraries/dph/dph-prim-interface/dist-boot "rm" -rf libraries/dph/dph-prim-seq/dist-boot "rm" -rf libraries/dph/dph-prim-par/dist-boot "rm" -rf libraries/dph/dph-lifted-base/dist-boot "rm" -rf libraries/dph/dph-lifted-boxed/dist-boot "rm" -rf libraries/dph/dph-lifted-copy/dist-boot "rm" -rf libraries/dph/dph-lifted-vseg/dist-boot "rm" -rf libraries/ghc-boot-th/dist libraries/ghc-boot/dist libraries/ghci/dist libraries/base/dist libraries/ghc-prim/dist libraries/integer-gmp/dist libraries/integer-simple/dist libraries/template-haskell/dist libraries/array/dist libraries/binary/dist libraries/bytestring/dist libraries/Cabal/Cabal/dist libraries/ghc-compact/dist libraries/containers/dist libraries/deepseq/dist libraries/directory/dist libraries/filepath/dist libraries/haskeline/dist libraries/hpc/dist libraries/mtl/dist libraries/parsec/dist libraries/pretty/dist libraries/process/dist libraries/terminfo/dist libraries/text/dist libraries/time/dist libraries/transformers/dist libraries/unix/dist libraries/Win32/dist libraries/xhtml/dist libraries/parallel/dist libraries/stm/dist libraries/random/dist libraries/primitive/dist libraries/vector/dist libraries/dph/dph-base/dist libraries/dph/dph-prim-interface/dist libraries/dph/dph-prim-seq/dist libraries/dph/dph-prim-par/dist libraries/dph/dph-lifted-base/dist libraries/dph/dph-lifted-boxed/dist libraries/dph/dph-lifted-copy/dist libraries/dph/dph-lifted-vseg/dist "rm" -f libraries/base/base.buildinfo libraries/integer-gmp/integer-gmp.buildinfo libraries/terminfo/terminfo.buildinfo libraries/unix/unix.buildinfo "rm" -f libraries/ghc-boot-th/config.log libraries/ghc-boot/config.log libraries/ghci/config.log libraries/base/config.log libraries/ghc-prim/config.log libraries/integer-gmp/config.log libraries/integer-simple/config.log libraries/template-haskell/config.log libraries/array/config.log libraries/binary/config.log libraries/bytestring/config.log libraries/Cabal/Cabal/config.log libraries/ghc-compact/config.log libraries/containers/config.log libraries/deepseq/config.log libraries/directory/config.log libraries/filepath/config.log libraries/haskeline/config.log libraries/hpc/config.log libraries/mtl/config.log libraries/parsec/config.log libraries/pretty/config.log libraries/process/config.log libraries/terminfo/config.log libraries/text/config.log libraries/time/config.log libraries/transformers/config.log libraries/unix/config.log libraries/Win32/config.log libraries/xhtml/config.log libraries/parallel/config.log libraries/stm/config.log libraries/random/config.log libraries/primitive/config.log libraries/vector/config.log libraries/dph/dph-base/config.log libraries/dph/dph-prim-interface/config.log libraries/dph/dph-prim-seq/config.log libraries/dph/dph-prim-par/config.log libraries/dph/dph-lifted-base/config.log libraries/dph/dph-lifted-boxed/config.log libraries/dph/dph-lifted-copy/config.log libraries/dph/dph-lifted-vseg/config.log "rm" -f libraries/ghc-boot-th/config.status libraries/ghc-boot/config.status libraries/ghci/config.status libraries/base/config.status libraries/ghc-prim/config.status libraries/integer-gmp/config.status libraries/integer-simple/config.status libraries/template-haskell/config.status libraries/array/config.status libraries/binary/config.status libraries/bytestring/config.status libraries/Cabal/Cabal/config.status libraries/ghc-compact/config.status libraries/containers/config.status libraries/deepseq/config.status libraries/directory/config.status libraries/filepath/config.status libraries/haskeline/config.status libraries/hpc/config.status libraries/mtl/config.status libraries/parsec/config.status libraries/pretty/config.status libraries/process/config.status libraries/terminfo/config.status libraries/text/config.status libraries/time/config.status libraries/transformers/config.status libraries/unix/config.status libraries/Win32/config.status libraries/xhtml/config.status libraries/parallel/config.status libraries/stm/config.status libraries/random/config.status libraries/primitive/config.status libraries/vector/config.status libraries/dph/dph-base/config.status libraries/dph/dph-prim-interface/config.status libraries/dph/dph-prim-seq/config.status libraries/dph/dph-prim-par/config.status libraries/dph/dph-lifted-base/config.status libraries/dph/dph-lifted-boxed/config.status libraries/dph/dph-lifted-copy/config.status libraries/dph/dph-lifted-vseg/config.status "rm" -f libraries/base/include/HsBaseConfig.h libraries/process/include/HsProcessConfig.h libraries/unix/include/HsUnixConfig.h "rm" -rf libraries/dist-haddock "rm" -rf bindistprep/ test ! -d testsuite || make -C testsuite clean make[1]: Entering directory '/stuff3/work/GHC2/ghc/testsuite' make -C ./timeout clean make[2]: Entering directory '/stuff3/work/GHC2/ghc/testsuite/timeout' test ! -f Setup || ./Setup clean rm -f -rf install-inplace rm -f -f calibrate.out rm -f -f Setup Setup.exe Setup.hi Setup.o make[2]: Leaving directory '/stuff3/work/GHC2/ghc/testsuite/timeout' rm -f -f mk/*.o rm -f -f mk/*.hi rm -f -f mk/ghcconfig*.mk rm -f -f mk/ghc-config mk/ghc-config.exe rm -f -f driver/*.pyc make[1]: Leaving directory '/stuff3/work/GHC2/ghc/testsuite' real 0m50.990s user 0m0.496s sys 0m1.582s --