Urk! I expected Hadrian to be faster because it has more accurate dependencies. Simon | -----Original Message----- | From: ghc-devs [mailto:ghc-devs-bounces@haskell.org] On Behalf Of | Herbert Valerio Riedel | Sent: 17 November 2017 13:08 | To: ghc-devs@haskell.org | Subject: Observation on Hadrian's relative performance re current | buildsystem | | Hello GHC devs, | | I took the opportunity to give Hadrian a test-run to see whether it | could live up to the big promise of delivering a "more scalable, | faster" | system than the current GNU Make based system. Unfortunately, my | preliminary results don't back this claim, and actually make Hadrian | appear to be significantly slower. | | ---- | | Here's the summary of the results: | | | Hadrian | GNU Make | | +------------------------+-----------+-----------+ | | Compiling `hadrian` | 5m25s | 0 | | | (one-time setup) | | | | +------------------------+-----------+-----------+ | | build "all" at -j8 | 38m | 33m | | +------------------------+-----------+-----------+ | | no-op build at -j8 | 10.977s | 3.258s | | +------------------------+-----------+-----------+ | | "clean" | 21s | 51s | | +------------------------+-----------+-----------+ | | | So, Hadrian is ~5 minutes than GNU Make (or even ~10 minutes if you | also count the one-time setup cost). | | And what I personally consider a bit annoying is that it's ~3 times | slower detecting; i.e. you have to wait 11s for Hadrian to detect | there's nothing to be done which compared to GNU Make (which currently | needlessly re-runs Sphinx; so it could be even faster!) is very | noticeable to me. | | There's a silver-lining though, deleting files is the part which is a | lot more costly in the GNU Make system currently since artifacts are | spread over several (scroll to the end of this email) subfolders | there. Whereas Hadrian did something we should have done for the GNU | Make system as well (and probably would have done sooner or later | anyway in order to support the srcdir!=buildir scheme that people are | used to from GNU Autotools projects); Hadrian places build-artifacts | into a few top-level folders, and so cleaning up is trivial and | requires to unlink only a few folders from the filesystem. | | ---- | | At the very least, I'd expect Hadrian to be as fast as the GNU Make | system (and ideally beat it, not the least as this was besides | maintainability its big promise), but so far it doesn't seem to | deliver that promise for me. | | It could easily be that I'm comparing apples to oranges here or that | I've otherwise overlooked something, so let me describe how I came to | this conclusion: | | I tried this on an reasonably idle Linux workstation with an Intel(R) | Core(TM) i7-3770 CPU @ 3.40GHz CPU, and with 32GiB RAM (i.e. the | filesystem content was well-cached into memory; NB: a ghc source tree | + compiled artifacts takes up about 4GiB on the filesystem). | | I've started from a fresh Git clone, i.e. | | git clone --recursive git://git.haskell.org/ghc.git | | followed by | | ./boot | ./configure | | At this point, we're at the common point from which both the Hadrian | and the GNU Make build-system would start diverging: | | For the Hadrian build-system, we need to pay for a one-time setup, | since we need to build the `hadrian` executable (which requires to | build the in-tree lib:Cabal as an in-place library): | | | $ time ./hadrian/build.sh --help | | ... | | real 5m25.992s | user 6m19.196s | sys 0m6.079s | | | I'm not too worried about this part, as there's a few tricks by which | we could likely bring that down to about 2 minutes or so, and we | mostly pay this setup-cost, when lib:Cabal and/or `hadrian` changes | and requires to be recompiled. | | | Now, after having made sure that the `hadrian` executable is fresh, I | started the actual build: | | $ time ./hadrian/build.sh -j8 | | ... | | shakeArgsWith 0.000s 0% | Function shake 0.178s 0% | Database read 0.000s 0% | With database 0.000s 0% | Running rules 2338.398s 99% | ========================= | Pool finished (5261 threads, 8 max) 0.002s 0% | Lint checking 0.111s 0% | Total 2338.690s 100% | Build completed in 38:59m | | real 38m59.626s | user 219m7.421s | sys 11m7.584s | | | then I immediately re-issued the same command to test how long it | takes to perform a no-op build: | | | $ time ./hadrian/build.sh -j8 | | Up to date | Up to date | shakeArgsWith 0.000s 0% | Function shake 0.183s 1% | Database read 0.144s 1% | With database 0.241s 2% | Running rules 9.379s 93% | ========================= | Pool finished (4165 threads, 8 max) 0.004s 0% | Lint checking 0.101s 1% | Total 10.051s 100% | Build completed in 0:11m | | | real 0m10.977s | user 0m19.399s | sys 0m2.443s | | | Same without -j8: | | $ time ./hadrian/build.sh | Up to date | Up to date | shakeArgsWith 0.000s 0% | Function shake 0.175s 2% | Database read 0.123s 1% | With database 0.197s 2% | Running rules 7.710s 92% | ========================= | Pool finished (1 threads, 1 max) 0.003s 0% | Lint checking 0.084s 1% | Total 8.293s 100% | Build completed in 0:09m | | | real 0m9.196s | user 0m8.656s | sys 0m0.724s | | | And finally clean it again: | | $ sync | $ time ./hadrian/build.sh clean | Up to date | Up to date | | Remove directory _build/stage0 | | Remove directory _build/stage1 | | Remove directory _build/stage2 | | Remove directory _build/stage3 | | Remove directory inplace/bin | | Remove directory inplace/lib | | Remove directory sdistprep | | Remove Hadrian files... | | Remove directory _build/generated | | Done. | shakeArgsWith 0.000s 0% | Function shake 0.175s 0% | Database read 0.114s 0% | With database 0.205s 0% | Running rules 20.037s 97% | ======================== | Pool finished (1 threads, 1 max) 0.002s 0% | Lint checking 0.006s 0% | Total 20.540s 100% | Build completed in 0:21m | | | real 0m21.426s | user 0m1.415s | sys 0m1.045s | | | | ---- | | Running the full build via our rusty GNU Make system: | | $ time make V=0 -j8 | real 33m30.801s | user 157m49.520s | sys 6m49.289s | | A no-op build: | | $ time make V=0 -j8 | ===--- building phase 0 | make --no-print-directory -f ghc.mk phase=0 phase_0_builds | make[1]: Nothing to be done for 'phase_0_builds'. | ===--- building phase 1 | make --no-print-directory -f ghc.mk phase=1 phase_1_builds | make[1]: Nothing to be done for 'phase_1_builds'. | ===--- building final phase | make --no-print-directory -f ghc.mk phase=final all | make -C utils/haddock/doc html SPHINX_BUILD=/usr/bin/sphinx-build | /usr/bin/sphinx-build -b html . .build-html | Running Sphinx v1.3.6 | loading translations [en]... done | loading pickled environment... done | building [mo]: targets for 0 po files that are out of date | building [html]: targets for 0 source files that are out of date | updating environment: 0 added, 0 changed, 0 removed | looking for now-outdated files... none found | no targets are out of date. | build succeeded. | cp -R utils/haddock/doc/.build-html utils/haddock/doc/haddock | | real 0m3.258s | user 0m3.134s | sys 0m0.283s | | And finally `make clean`: | | $ sync | $ time make clean | make --no-print-directory -f ghc.mk clean CLEANING=YES | "rm" -rf inplace/bin inplace/lib | "rm" -rf utils/touchy/dist | "rm" -rf inplace/lib/bin/touchy | "rm" -rf utils/unlit/dist | "rm" -rf inplace/lib/bin/unlit | "rm" -rf utils/unlit/dist-install | "rm" -rf utils/hp2ps/dist | "rm" -rf inplace/bin/hp2ps | "rm" -rf inplace/lib/bin/hp2ps | "rm" -rf utils/hp2ps/dist-install | "rm" -rf driver/split/dist inplace/lib/bin/ghc-split | "rm" -rf utils/genprimopcode/dist | "rm" -rf inplace/bin/genprimopcode | "rm" -rf libffi/build libffi/stamp.ffi.static-shared.configure | libffi/stamp.ffi.static-shared.build libffi/stamp.ffi.static- | shared.install libffi/dist-install | "rm" -rf utils/deriveConstants/dist | "rm" -rf inplace/bin/deriveConstants | "rm" -rf includes/ghcautoconf.h includes/ghcplatform.h | includes/ghcversion.h | "rm" -rf rts/dist | "rm" -rf bindisttest/"install dir" bindisttest/HelloWorld | bindisttest/HelloWorld.o bindisttest/HelloWorld.hi bindisttest/output | "rm" -rf utils/genapply/dist | "rm" -rf inplace/bin/genapply | "rm" -rf libraries/integer-gmp/include/ghc-gmp.h | libraries/integer-gmp/gmp/config.mk libraries/integer-gmp/gmp/libgmp.a | libraries/integer-gmp/gmp/gmp.h libraries/integer-gmp/gmp/gmpbuild | libraries/integer-gmp/gmp/gmp-6.1.2 | "rm" -rf libraries/integer-gmp/gmp/objs | "rm" -rf libraries/integer-gmp/gmp/gmpbuild | "rm" -rf utils/haddock/dist | "rm" -rf inplace/bin/haddock | "rm" -rf inplace/lib/bin/haddock | "rm" -rf compiler/stage1 | "rm" -rf compiler/stage2 | "rm" -rf compiler/stage3 | "rm" -rf utils/hsc2hs/dist | "rm" -rf inplace/bin/hsc2hs | "rm" -rf inplace/lib/bin/hsc2hs | "rm" -rf utils/hsc2hs/dist-install | "rm" -rf utils/ghc-pkg/dist | "rm" -rf inplace/bin/ghc-pkg | "rm" -rf inplace/lib/bin/ghc-pkg | "rm" -rf utils/ghc-pkg/dist-install | "rm" -rf utils/ghctags/dist-install | "rm" -rf inplace/bin/ghctags | "rm" -rf utils/check-api-annotations/dist-install | "rm" -rf inplace/bin/check-api-annotations | "rm" -rf utils/check-ppr/dist-install | "rm" -rf inplace/bin/check-ppr | "rm" -rf utils/ghc-cabal/dist bootstrapping | "rm" -rf utils/ghc-cabal/dist-install | "rm" -rf utils/hpc/dist-install | "rm" -rf inplace/bin/hpc | "rm" -rf inplace/lib/bin/hpc | "rm" -rf utils/runghc/dist-install | "rm" -rf inplace/bin/runghc | "rm" -rf inplace/lib/bin/runghc | "rm" -rf ghc/stage1 | "rm" -rf inplace/bin/ghc-stage1 | "rm" -rf inplace/lib/bin/ghc-stage1 | "rm" -rf ghc/stage2 | "rm" -rf inplace/bin/ghc-stage2 | "rm" -rf inplace/lib/bin/ghc-stage2 | "rm" -rf ghc/stage3 | "rm" -rf docs/users_guide/.doctrees-html/ | docs/users_guide/.doctrees-pdf/ docs/users_guide/build-html/ | docs/users_guide/build-pdf/ docs/users_guide/users_guide.pdf | "rm" -rf docs/users_guide/.doctrees-man/ docs/users_guide/build- | man/ | "rm" -rf utils/count_lines/dist inplace/bin/count_lines | "rm" -rf utils/compare_sizes/dist-install | "rm" -rf iserv/stage2 | "rm" -rf inplace/lib/bin/ghc-iserv | "rm" -rf iserv/stage2_p | "rm" -rf inplace/lib/bin/ghc-iserv-prof | "rm" -rf iserv/stage2_dyn | "rm" -rf inplace/lib/bin/ghc-iserv-dyn | "rm" -f libraries/integer-gmp/include/HsIntegerGmp.h | libraries/base/include/EventConfig.h mk/config.mk.old | mk/project.mk.old compiler/ghc.cabal.old includes/GHCConstants.h | includes/DerivedConstants.h includes/ghcautoconf.h | includes/ghcplatform.h includes/ghcversion.h utils/ghc-pkg/Version.hs | compiler/prelude/primops.txt | "rm" -rf includes/dist-derivedconstants | "rm" -rf inplace/bin | "rm" -rf inplace/lib | "rm" -rf libraries/bootstrapping.conf | "rm" -f mk/are-validating.mk | "rm" -rf libraries/ghc-boot-th/dist-install | "rm" -rf libraries/ghc-boot/dist-install | "rm" -rf libraries/ghci/dist-install | "rm" -rf libraries/base/dist-install | "rm" -rf libraries/ghc-prim/dist-install | "rm" -rf libraries/integer-gmp/dist-install | "rm" -rf libraries/integer-simple/dist-install | "rm" -rf libraries/template-haskell/dist-install | "rm" -rf libraries/array/dist-install | "rm" -rf libraries/binary/dist-install | "rm" -rf libraries/bytestring/dist-install | "rm" -rf libraries/Cabal/Cabal/dist-install | "rm" -rf libraries/ghc-compact/dist-install | "rm" -rf libraries/containers/dist-install | "rm" -rf libraries/deepseq/dist-install | "rm" -rf libraries/directory/dist-install | "rm" -rf libraries/filepath/dist-install | "rm" -rf libraries/haskeline/dist-install | "rm" -rf libraries/hpc/dist-install | "rm" -rf libraries/mtl/dist-install | "rm" -rf libraries/parsec/dist-install | "rm" -rf libraries/pretty/dist-install | "rm" -rf libraries/process/dist-install | "rm" -rf libraries/terminfo/dist-install | "rm" -rf libraries/text/dist-install | "rm" -rf libraries/time/dist-install | "rm" -rf libraries/transformers/dist-install | "rm" -rf libraries/unix/dist-install | "rm" -rf libraries/Win32/dist-install | "rm" -rf libraries/xhtml/dist-install | "rm" -rf libraries/parallel/dist-install | "rm" -rf libraries/stm/dist-install | "rm" -rf libraries/random/dist-install | "rm" -rf libraries/primitive/dist-install | "rm" -rf libraries/vector/dist-install | "rm" -rf libraries/dph/dph-base/dist-install | "rm" -rf libraries/dph/dph-prim-interface/dist-install | "rm" -rf libraries/dph/dph-prim-seq/dist-install | "rm" -rf libraries/dph/dph-prim-par/dist-install | "rm" -rf libraries/dph/dph-lifted-base/dist-install | "rm" -rf libraries/dph/dph-lifted-boxed/dist-install | "rm" -rf libraries/dph/dph-lifted-copy/dist-install | "rm" -rf libraries/dph/dph-lifted-vseg/dist-install | "rm" -rf libraries/ghc-boot-th/dist-boot | "rm" -rf libraries/ghc-boot/dist-boot | "rm" -rf libraries/ghci/dist-boot | "rm" -rf libraries/base/dist-boot | "rm" -rf libraries/ghc-prim/dist-boot | "rm" -rf libraries/integer-gmp/dist-boot | "rm" -rf libraries/integer-simple/dist-boot | "rm" -rf libraries/template-haskell/dist-boot | "rm" -rf libraries/array/dist-boot | "rm" -rf libraries/binary/dist-boot | "rm" -rf libraries/bytestring/dist-boot | "rm" -rf libraries/Cabal/Cabal/dist-boot | "rm" -rf libraries/ghc-compact/dist-boot | "rm" -rf libraries/containers/dist-boot | "rm" -rf libraries/deepseq/dist-boot | "rm" -rf libraries/directory/dist-boot | "rm" -rf libraries/filepath/dist-boot | "rm" -rf libraries/haskeline/dist-boot | "rm" -rf libraries/hpc/dist-boot | "rm" -rf libraries/mtl/dist-boot | "rm" -rf libraries/parsec/dist-boot | "rm" -rf libraries/pretty/dist-boot | "rm" -rf libraries/process/dist-boot | "rm" -rf libraries/terminfo/dist-boot | "rm" -rf libraries/text/dist-boot | "rm" -rf libraries/time/dist-boot | "rm" -rf libraries/transformers/dist-boot | "rm" -rf libraries/unix/dist-boot | "rm" -rf libraries/Win32/dist-boot | "rm" -rf libraries/xhtml/dist-boot | "rm" -rf libraries/parallel/dist-boot | "rm" -rf libraries/stm/dist-boot | "rm" -rf libraries/random/dist-boot | "rm" -rf libraries/primitive/dist-boot | "rm" -rf libraries/vector/dist-boot | "rm" -rf libraries/dph/dph-base/dist-boot | "rm" -rf libraries/dph/dph-prim-interface/dist-boot | "rm" -rf libraries/dph/dph-prim-seq/dist-boot | "rm" -rf libraries/dph/dph-prim-par/dist-boot | "rm" -rf libraries/dph/dph-lifted-base/dist-boot | "rm" -rf libraries/dph/dph-lifted-boxed/dist-boot | "rm" -rf libraries/dph/dph-lifted-copy/dist-boot | "rm" -rf libraries/dph/dph-lifted-vseg/dist-boot | "rm" -rf libraries/ghc-boot-th/dist libraries/ghc-boot/dist | libraries/ghci/dist libraries/base/dist libraries/ghc-prim/dist | libraries/integer-gmp/dist libraries/integer-simple/dist | libraries/template-haskell/dist libraries/array/dist | libraries/binary/dist libraries/bytestring/dist | libraries/Cabal/Cabal/dist libraries/ghc-compact/dist | libraries/containers/dist libraries/deepseq/dist | libraries/directory/dist libraries/filepath/dist | libraries/haskeline/dist libraries/hpc/dist libraries/mtl/dist | libraries/parsec/dist libraries/pretty/dist libraries/process/dist | libraries/terminfo/dist libraries/text/dist libraries/time/dist | libraries/transformers/dist libraries/unix/dist libraries/Win32/dist | libraries/xhtml/dist libraries/parallel/dist libraries/stm/dist | libraries/random/dist libraries/primitive/dist libraries/vector/dist | libraries/dph/dph-base/dist libraries/dph/dph-prim-interface/dist | libraries/dph/dph-prim-seq/dist libraries/dph/dph-prim-par/dist | libraries/dph/dph-lifted-base/dist libraries/dph/dph-lifted- | boxed/dist libraries/dph/dph-lifted-copy/dist libraries/dph/dph- | lifted-vseg/dist | "rm" -f libraries/base/base.buildinfo libraries/integer- | gmp/integer-gmp.buildinfo libraries/terminfo/terminfo.buildinfo | libraries/unix/unix.buildinfo | "rm" -f libraries/ghc-boot-th/config.log libraries/ghc- | boot/config.log libraries/ghci/config.log libraries/base/config.log | libraries/ghc-prim/config.log libraries/integer-gmp/config.log | libraries/integer-simple/config.log libraries/template- | haskell/config.log libraries/array/config.log | libraries/binary/config.log libraries/bytestring/config.log | libraries/Cabal/Cabal/config.log libraries/ghc-compact/config.log | libraries/containers/config.log libraries/deepseq/config.log | libraries/directory/config.log libraries/filepath/config.log | libraries/haskeline/config.log libraries/hpc/config.log | libraries/mtl/config.log libraries/parsec/config.log | libraries/pretty/config.log libraries/process/config.log | libraries/terminfo/config.log libraries/text/config.log | libraries/time/config.log libraries/transformers/config.log | libraries/unix/config.log libraries/Win32/config.log | libraries/xhtml/config.log libraries/parallel/config.log | libraries/stm/config.log libraries/random/config.log | libraries/primitive/config.log libraries/vector/config.log | libraries/dph/dph-base/config.log libraries/dph/dph-prim- | interface/config.log libraries/dph/dph-prim-seq/config.log | libraries/dph/dph-prim-par/config.log libraries/dph/dph-lifted- | base/config.log libraries/dph/dph-lifted-boxed/config.log | libraries/dph/dph-lifted-copy/config.log libraries/dph/dph-lifted- | vseg/config.log | "rm" -f libraries/ghc-boot-th/config.status libraries/ghc- | boot/config.status libraries/ghci/config.status | libraries/base/config.status libraries/ghc-prim/config.status | libraries/integer-gmp/config.status libraries/integer- | simple/config.status libraries/template-haskell/config.status | libraries/array/config.status libraries/binary/config.status | libraries/bytestring/config.status | libraries/Cabal/Cabal/config.status libraries/ghc- | compact/config.status libraries/containers/config.status | libraries/deepseq/config.status libraries/directory/config.status | libraries/filepath/config.status libraries/haskeline/config.status | libraries/hpc/config.status libraries/mtl/config.status | libraries/parsec/config.status libraries/pretty/config.status | libraries/process/config.status libraries/terminfo/config.status | libraries/text/config.status libraries/time/config.status | libraries/transformers/config.status libraries/unix/config.status | libraries/Win32/config.status libraries/xhtml/config.status | libraries/parallel/config.status libraries/stm/config.status | libraries/random/config.status libraries/primitive/config.status | libraries/vector/config.status libraries/dph/dph-base/config.status | libraries/dph/dph-prim-interface/config.status libraries/dph/dph- | prim-seq/config.status libraries/dph/dph-prim-par/config.status | libraries/dph/dph-lifted-base/config.status libraries/dph/dph-lifted- | boxed/config.status libraries/dph/dph-lifted-copy/config.status | libraries/dph/dph-lifted-vseg/config.status | "rm" -f libraries/base/include/HsBaseConfig.h | libraries/process/include/HsProcessConfig.h | libraries/unix/include/HsUnixConfig.h | "rm" -rf libraries/dist-haddock | "rm" -rf bindistprep/ | test ! -d testsuite || make -C testsuite clean | make[1]: Entering directory '/stuff3/work/GHC2/ghc/testsuite' | make -C ./timeout clean | make[2]: Entering directory | '/stuff3/work/GHC2/ghc/testsuite/timeout' | test ! -f Setup || ./Setup clean | rm -f -rf install-inplace | rm -f -f calibrate.out | rm -f -f Setup Setup.exe Setup.hi Setup.o | make[2]: Leaving directory | '/stuff3/work/GHC2/ghc/testsuite/timeout' | rm -f -f mk/*.o | rm -f -f mk/*.hi | rm -f -f mk/ghcconfig*.mk | rm -f -f mk/ghc-config mk/ghc-config.exe | rm -f -f driver/*.pyc | make[1]: Leaving directory '/stuff3/work/GHC2/ghc/testsuite' | | real 0m50.990s | user 0m0.496s | sys 0m1.582s | | | | -- | _______________________________________________ | ghc-devs mailing list | ghc-devs@haskell.org | https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmail.h | askell.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fghc- | devs&data=02%7C01%7Csimonpj%40microsoft.com%7Cec692486529544de389e08d5 | 2dbc7432%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6364652097984492 | 46&sdata=I1TY6c2l59jou87OVjqScDvct%2FaygyM4HRuSirCSp5w%3D&reserved=0