Matt has access to the M1 builder in my closet now. The darwin performance issue
is mainly there since BigSur, and (afaik) primarily due to the amount of DYLD_LIBRARY_PATH's
we pass to GHC invocations. The system linker spends the majority of the time in the
kernel stat'ing and getelements (or some similar directory) call for each and every possible
path.
Switching to hadrian will cut down the time from ~5hs to ~2hs. At some point we had make
builds <90min by just killing all DYLD_LIBRARY_PATH logic we ever had, but that broke
bindists.
The CI has time values attached and some summary at the end right now, which highlights
time spent in the system and in user mode. This is up to 80% sys, 20% user, and went to
something like 20% sys, 80% user after nuking all DYLD_LIBRARY_PATH's, with hadrian it's
closer to ~25% sys, 75% user.
Of note, this is mostly due to time spent during the *test-suite*, not the actual build. For the
actual build make and hadrian are comparable, though I've seen hadrian to oddly have a
much higher variance in how long it takes to *build* ghc, whereas the make build was more
consistent.
The test-suite quite notoriously calls GHC *a lot of times*, which makes any linker issue due
to DYLD_LIBRARY_PATH (and similar lookups) much worse.
If we would finally split building and testing, we'd see this more clearly I believe. Maybe this
is motivation enough for someone to come forward to break build/test into two CI steps?
Cheers,
Moritz