Hi Ben,
For what it's worth, some of us have recognized for quite some time that BFD ld is a known slow spot in the compilation pipeline. However, up until now we have considered this to be more of an issue to be handled at the packaging level than in GHC issue. Currently, GHC relies on `gcc` to link and gcc has its own idea of the default linker. Many users (e.g. Debian packagers, users who are cross compiling) are quite unhappy when GHC coerces the system toolchain into changing its default behavior.
Consequently we have maintained the status quo and silently wished that Linux distributions would start moving to gold by default. Sadly there continues to be little progress on this matter.
This is a great point. I don't know if using BFD ld vs gold is a controversial move, given that they are both GNU projects. I would imagine if lld was in the game, that might be more controversial. I have the theory that both Haskell packagers and Linux distributions are simply conservative about changing the linker not because they don't want it, but because they are scared of breaking people's code, and especially in the packagers' case, that they don't feel expert enough to judge whether them making this move will be safe. I can imagine that they would rather welcome if the GHC devs made this decision, since after all they generate the linker invocations and have the knowledge to judge whether they are supposed to work. If a packager wants to *override* GHC's default behaviour, that should certainly be possible, but as a packager I would find it unfair if GHC pushed the burden of setting up the optimal defaults for my users on me. For a comparison with other programming languages: From tickets like https://github.com/golang/go/issues/1588 seems that go also uses gold by default when available. Rust switched to gold by default with https://github.com/rust-lang/rust/pull/29974, but reverted the commit with https://github.com/rust-lang/rust/pull/30913, with reasons being the linked tickets. The Rust tickets in particular have already discussed the various issues we may be running into regarding autodetection, cross compilation and so on, it may be good to get inspiration from there.
* Do we want to also enable it in source distributions as well? If yes then developers may be caught by surprise
I would recommend to do the same in source and binary distributions. The surprise should be minimised by properly advertising it as a key thing in the release notes. Packagers are supposed to read those, and in fact and most users of GHC love to read them for they contain many goodies. Another option is to not change the default or enable autodetection, but make it trivial to use gold for the user, e.g. by at least fixing the Cabal library (currently you need to put your linker options into a lot of places to get the desired results, see https://github.com/haskell/cabal/issues/4435, and even if you do, for some invocations Cabal just doesn't pass on what you tell it to do, see https://github.com/haskell/cabal/issues/4439). We could run this for a GHC release term, and also have cabal-install and stack recommend people that they should try to set some easy option to use the gold linker if available. If that works well, we could switch to autodetection the release after that. I haven't though much about whether this is actually a better idea than just enabling it where we can. Niklas