On 10/20/23 04:00, Simon Peyton Jones wrote:

A very large proportion of libraries, and virtually all end-user applications, transitively depend on Template Haskell. Whether they use Template Haskell directly or not. So if we're saying “base is reinstallable, except when you have Template Haskell somewhere”, we're effectively saying “base is not reinstallable”. Now, it could be a good stepping-stone, from an engineering standpoint, but I don't think we could deliver this and be satisfied that we've accomplished anything.

No one has yet answered my naive question (from 3 days ago) asking why Template Haskell stops base being reinstallable.  I'll quote it here for completeness.

Let's say that
  • An old library mylib (which uses TH) depends on base-4.7.
  • A new GHC, say GHC 9.10, depends on a newer version of base-4.9, which in turn depends on ghc-internal-9.10.
  • At the same time, though, we release base-4.7.1, which depends on ghc-internal-9.10, and exposes the base-4.7 API.
At this point we use ghc-9.10 to compile L, against base-4.7.1.   (Note that the ghc-9.10 binary includes a compiled form of `base-4.9`.)
  • That produces compiled object files, such as, mylib:M.o.  
  • To run TH we need to link them with the running binary
  • So we need to link the compiled `base-4.7.1` as well.  No problem: it contains very little code; it is mostly a shim for ghc-internal-9.10
So the only thing we need is the ability to have a single linked binary that includes (the compiled form for) two different versions/instantiations of `base`.   I think that's already supported: each has a distinct "installed package id".
(End of quote)

What am I missing?

Simon

Simon I think you are right on the level of GHC itself: GHC can indeed cope with multiple versions of libraries just fine. However if we do this we run the risk of the user getting "base.x.y FooType is not the same as base.x.(y+1) FooType" errors, and I don't think that is good. cabal-install and stack current enforce build plans such that those sorts of errors are not possible, and I think that is a good thing we should not revisit at this time.

I don't think this should stop anything about reinstallable base, however. We just need to make a nice error when someone tries to splice in code using the wrong version of template haskell (or otherwise interact with GHC) using ABI hashes. Then we have the following situation:

  1. Users that use stack/cabal-install plans only get one version of base, and if they have template-haskell using the the "wrong" version of base (identified via API hashes) they will get a nice error.
  2. Users that "go rogue" and manually set up their project to mix base versions and avoid other errors will have their TH work as you describe./

So the thing Simon talks about is not encouraged, but it is a "free feature" we shouldn't go out of our way to prevent either.

John